Sep 10 12:15:49.896814 ip-10-0-130-221 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 10 12:15:49.896829 ip-10-0-130-221 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 10 12:15:49.896854 ip-10-0-130-221 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 10 12:15:49.897142 ip-10-0-130-221 systemd[1]: Failed to start Kubernetes Kubelet. Sep 10 12:15:59.918474 ip-10-0-130-221 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 10 12:15:59.918492 ip-10-0-130-221 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 5ba18ba3460b4ef79cf1fd1a47853f03 -- Sep 10 12:17:59.871256 ip-10-0-130-221 systemd[1]: Starting Kubernetes Kubelet... Sep 10 12:18:00.299714 ip-10-0-130-221 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 12:18:00.299714 ip-10-0-130-221 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 10 12:18:00.299714 ip-10-0-130-221 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 12:18:00.299714 ip-10-0-130-221 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 10 12:18:00.299714 ip-10-0-130-221 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 12:18:00.301963 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.301832 2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 10 12:18:00.306006 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.305982 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 12:18:00.306006 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306006 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 12:18:00.306093 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306010 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 12:18:00.306093 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306014 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 12:18:00.306093 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306018 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 12:18:00.306093 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306020 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 12:18:00.306093 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306023 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 12:18:00.306093 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306026 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 12:18:00.306093 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306029 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 12:18:00.306093 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306032 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 12:18:00.306093 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306035 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 12:18:00.306093 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306037 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 12:18:00.306093 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306040 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 12:18:00.306093 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306042 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 12:18:00.306093 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306045 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 12:18:00.306093 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306048 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 12:18:00.306093 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306052 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 12:18:00.306093 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306054 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 12:18:00.306093 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306057 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 12:18:00.306093 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306060 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 12:18:00.306093 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306062 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 12:18:00.306093 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306065 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 10 12:18:00.306581 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306068 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 12:18:00.306581 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306070 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 12:18:00.306581 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306074 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 12:18:00.306581 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306077 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 12:18:00.306581 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306079 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 12:18:00.306581 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306082 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 12:18:00.306581 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306085 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 12:18:00.306581 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306088 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 12:18:00.306581 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306090 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 12:18:00.306581 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306093 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 12:18:00.306581 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306097 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 12:18:00.306581 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306101 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 12:18:00.306581 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306105 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 12:18:00.306581 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306108 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 12:18:00.306581 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306111 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 12:18:00.306581 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306114 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 12:18:00.306581 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306117 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 12:18:00.306581 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306120 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 12:18:00.306581 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306123 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 12:18:00.307066 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306126 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 12:18:00.307066 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306129 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 12:18:00.307066 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306132 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 12:18:00.307066 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306135 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 12:18:00.307066 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306137 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 12:18:00.307066 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306140 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 12:18:00.307066 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306142 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 12:18:00.307066 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306145 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 12:18:00.307066 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306147 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 12:18:00.307066 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306150 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 12:18:00.307066 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306153 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 12:18:00.307066 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306155 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 12:18:00.307066 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306158 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 12:18:00.307066 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306161 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 12:18:00.307066 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306166 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 12:18:00.307066 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306169 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 12:18:00.307066 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306171 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 12:18:00.307066 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306174 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 12:18:00.307066 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306177 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 12:18:00.307066 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306180 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 12:18:00.307575 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306183 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 12:18:00.307575 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306187 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 12:18:00.307575 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306190 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 12:18:00.307575 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306193 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 12:18:00.307575 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306195 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 12:18:00.307575 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306198 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 12:18:00.307575 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306201 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 12:18:00.307575 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306204 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 12:18:00.307575 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306206 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 12:18:00.307575 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306209 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 12:18:00.307575 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306211 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 12:18:00.307575 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306214 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 12:18:00.307575 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306217 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 12:18:00.307575 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306219 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 12:18:00.307575 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306222 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 12:18:00.307575 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306224 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 12:18:00.307575 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306227 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 12:18:00.307575 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306230 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 12:18:00.307575 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306232 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 12:18:00.308122 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306234 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 12:18:00.308122 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306237 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 12:18:00.308122 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306240 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 12:18:00.308122 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306243 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 12:18:00.308122 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306246 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 12:18:00.308122 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306249 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 12:18:00.308122 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306251 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 12:18:00.308122 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306256 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 12:18:00.308122 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306259 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 12:18:00.308122 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306262 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 12:18:00.308122 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306264 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 12:18:00.308122 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306267 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 12:18:00.308122 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306270 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 12:18:00.308122 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306273 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 12:18:00.308122 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306277 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 12:18:00.308122 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306280 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 12:18:00.308122 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306284 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 12:18:00.308122 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306287 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 12:18:00.308122 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.306290 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 12:18:00.308586 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308577 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 12:18:00.308615 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308587 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 12:18:00.308615 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308590 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 12:18:00.308615 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308594 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 12:18:00.308615 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308598 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 12:18:00.308615 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308601 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 12:18:00.308615 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308606 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 12:18:00.308615 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308610 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 12:18:00.308615 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308614 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 12:18:00.308615 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308617 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 12:18:00.308615 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308620 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 12:18:00.308852 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308623 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 12:18:00.308852 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308627 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 12:18:00.308852 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308629 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 12:18:00.308852 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308633 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 12:18:00.308852 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308637 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 12:18:00.308852 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308640 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 12:18:00.308852 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308642 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 12:18:00.308852 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308645 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 12:18:00.308852 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308647 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 12:18:00.308852 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308650 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 12:18:00.308852 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308653 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 12:18:00.308852 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308656 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 12:18:00.308852 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308658 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 12:18:00.308852 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308661 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 12:18:00.308852 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308664 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 12:18:00.308852 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308666 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 12:18:00.308852 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308668 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 12:18:00.308852 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308671 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 12:18:00.308852 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308674 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 12:18:00.309366 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308676 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 12:18:00.309366 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308678 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 12:18:00.309366 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308681 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 12:18:00.309366 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308683 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 12:18:00.309366 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308686 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 12:18:00.309366 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308690 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 12:18:00.309366 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308692 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 12:18:00.309366 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308695 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 12:18:00.309366 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308698 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 12:18:00.309366 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308701 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 12:18:00.309366 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308704 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 12:18:00.309366 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308707 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 12:18:00.309366 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308709 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 12:18:00.309366 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308712 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 12:18:00.309366 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308715 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 12:18:00.309366 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308717 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 12:18:00.309366 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308720 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 12:18:00.309366 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308723 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 12:18:00.309366 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308725 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 12:18:00.309366 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308728 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 12:18:00.309873 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308730 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 12:18:00.309873 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308733 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 12:18:00.309873 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308736 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 12:18:00.309873 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308738 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 12:18:00.309873 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308740 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 12:18:00.309873 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308743 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 12:18:00.309873 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308746 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 12:18:00.309873 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308748 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 10 12:18:00.309873 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308751 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 12:18:00.309873 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308754 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 12:18:00.309873 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308756 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 12:18:00.309873 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308760 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 12:18:00.309873 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308762 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 12:18:00.309873 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308765 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 12:18:00.309873 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308768 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 12:18:00.309873 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308770 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 12:18:00.309873 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308773 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 12:18:00.309873 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308776 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 12:18:00.309873 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308778 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 12:18:00.309873 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308781 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 12:18:00.310380 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308785 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 12:18:00.310380 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308788 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 12:18:00.310380 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308791 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 12:18:00.310380 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308793 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 12:18:00.310380 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308795 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 12:18:00.310380 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308798 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 12:18:00.310380 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308801 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 12:18:00.310380 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308804 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 12:18:00.310380 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308806 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 12:18:00.310380 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308809 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 12:18:00.310380 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308811 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 12:18:00.310380 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308813 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 12:18:00.310380 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308816 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 12:18:00.310380 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308818 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 12:18:00.310380 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308821 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 12:18:00.310380 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308824 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 12:18:00.310380 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308826 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 12:18:00.310380 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308829 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 12:18:00.310380 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308831 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 12:18:00.310380 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308834 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 12:18:00.310886 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308837 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 12:18:00.310886 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308839 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 12:18:00.310886 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308843 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 12:18:00.310886 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308847 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 12:18:00.310886 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308850 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 12:18:00.310886 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308853 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 12:18:00.310886 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308871 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 12:18:00.310886 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308873 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 12:18:00.310886 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.308877 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 12:18:00.310886 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309370 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 12:18:00.310886 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309378 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 12:18:00.310886 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309382 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 12:18:00.310886 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309386 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 12:18:00.310886 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309389 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 12:18:00.310886 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309392 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 12:18:00.310886 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309395 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 12:18:00.310886 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309398 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 12:18:00.310886 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309401 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 12:18:00.310886 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309404 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 12:18:00.310886 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309407 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 12:18:00.311365 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309409 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 12:18:00.311365 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309412 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 12:18:00.311365 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309415 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 12:18:00.311365 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309418 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 12:18:00.311365 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309420 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 12:18:00.311365 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309423 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 12:18:00.311365 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309426 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 12:18:00.311365 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309429 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 12:18:00.311365 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309431 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 12:18:00.311365 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309434 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 12:18:00.311365 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309437 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 12:18:00.311365 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309440 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 12:18:00.311365 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309442 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 12:18:00.311365 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309445 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 12:18:00.311365 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309447 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 12:18:00.311365 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309451 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 12:18:00.311365 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309453 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 12:18:00.311365 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309456 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 12:18:00.311365 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309459 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 12:18:00.311365 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309462 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 12:18:00.311850 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309464 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 12:18:00.311850 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309467 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 12:18:00.311850 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309470 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 12:18:00.311850 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309472 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 12:18:00.311850 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309475 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 12:18:00.311850 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309478 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 12:18:00.311850 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309481 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 12:18:00.311850 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309483 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 12:18:00.311850 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309486 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 12:18:00.311850 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309489 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 12:18:00.311850 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309491 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 12:18:00.311850 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309494 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 12:18:00.311850 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309497 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 12:18:00.311850 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309500 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 12:18:00.311850 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309502 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 12:18:00.311850 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309507 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 12:18:00.311850 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309510 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 12:18:00.311850 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309513 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 12:18:00.311850 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309515 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 12:18:00.312355 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309518 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 12:18:00.312355 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309521 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 12:18:00.312355 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309523 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 12:18:00.312355 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309526 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 12:18:00.312355 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309529 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 12:18:00.312355 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309531 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 12:18:00.312355 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309534 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 12:18:00.312355 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309537 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 12:18:00.312355 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309540 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 12:18:00.312355 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309543 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 12:18:00.312355 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309545 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 12:18:00.312355 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309548 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 12:18:00.312355 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309551 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 12:18:00.312355 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309553 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 12:18:00.312355 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309556 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 12:18:00.312355 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309559 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 12:18:00.312355 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309562 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 12:18:00.312355 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309565 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 12:18:00.312355 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309568 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 12:18:00.312881 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309571 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 12:18:00.312881 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309574 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 12:18:00.312881 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309576 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 12:18:00.312881 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309579 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 12:18:00.312881 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309582 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 12:18:00.312881 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309584 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 12:18:00.312881 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309587 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 12:18:00.312881 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309590 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 12:18:00.312881 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309592 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 12:18:00.312881 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309596 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 12:18:00.312881 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309599 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 12:18:00.312881 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309602 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 12:18:00.312881 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309604 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 12:18:00.312881 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309608 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 12:18:00.312881 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309610 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 12:18:00.312881 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309613 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 12:18:00.312881 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309615 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 12:18:00.312881 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309619 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 12:18:00.312881 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309622 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 12:18:00.312881 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309625 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 12:18:00.313387 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309627 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 12:18:00.313387 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309631 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 12:18:00.313387 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309633 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 12:18:00.313387 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309636 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 12:18:00.313387 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309638 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 12:18:00.313387 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309641 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 12:18:00.313387 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309643 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 12:18:00.313387 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309646 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 12:18:00.313387 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309648 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 10 12:18:00.313387 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.309651 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 12:18:00.313387 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309734 2580 flags.go:64] FLAG: --address="0.0.0.0" Sep 10 12:18:00.313387 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309775 2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 10 12:18:00.313387 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309787 2580 flags.go:64] FLAG: --anonymous-auth="true" Sep 10 12:18:00.313387 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309792 2580 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 10 12:18:00.313387 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309797 2580 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 10 12:18:00.313387 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309801 2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 10 12:18:00.313387 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309806 2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 10 12:18:00.313387 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309811 2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 10 12:18:00.313387 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309815 2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 10 12:18:00.313387 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309818 2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 10 12:18:00.313387 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309821 2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 10 12:18:00.313387 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309825 2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 10 12:18:00.313952 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309829 2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 10 12:18:00.313952 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309833 2580 flags.go:64] FLAG: --cgroup-root="" Sep 10 12:18:00.313952 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309836 2580 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 10 12:18:00.313952 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309838 2580 flags.go:64] FLAG: --client-ca-file="" Sep 10 12:18:00.313952 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309842 2580 flags.go:64] FLAG: --cloud-provider="external" Sep 10 12:18:00.313952 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309845 2580 flags.go:64] FLAG: --cluster-dns="[]" Sep 10 12:18:00.313952 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309850 2580 flags.go:64] FLAG: --cluster-domain="" Sep 10 12:18:00.313952 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309866 2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 10 12:18:00.313952 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309869 2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 10 12:18:00.313952 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309873 2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 10 12:18:00.313952 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309877 2580 flags.go:64] FLAG: --container-log-max-files="5" Sep 10 12:18:00.313952 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309882 2580 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 10 12:18:00.313952 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309886 2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 10 12:18:00.313952 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309890 2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 10 12:18:00.313952 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309894 2580 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 10 12:18:00.313952 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309898 2580 flags.go:64] FLAG: --contention-profiling="false" Sep 10 12:18:00.313952 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309901 2580 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 10 12:18:00.313952 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309904 2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 10 12:18:00.313952 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309907 2580 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 10 12:18:00.313952 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309910 2580 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 10 12:18:00.313952 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309915 2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 10 12:18:00.313952 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309918 2580 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 10 12:18:00.313952 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309921 2580 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 10 12:18:00.313952 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309924 2580 flags.go:64] FLAG: --enable-load-reader="false" Sep 10 12:18:00.313952 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309928 2580 flags.go:64] FLAG: --enable-server="true" Sep 10 12:18:00.314562 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309935 2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 10 12:18:00.314562 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309940 2580 flags.go:64] FLAG: --event-burst="100" Sep 10 12:18:00.314562 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309944 2580 flags.go:64] FLAG: --event-qps="50" Sep 10 12:18:00.314562 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309947 2580 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 10 12:18:00.314562 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309950 2580 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 10 12:18:00.314562 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309953 2580 flags.go:64] FLAG: --eviction-hard="" Sep 10 12:18:00.314562 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309957 2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 10 12:18:00.314562 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309960 2580 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 10 12:18:00.314562 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309964 2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 10 12:18:00.314562 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309967 2580 flags.go:64] FLAG: --eviction-soft="" Sep 10 12:18:00.314562 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309970 2580 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 10 12:18:00.314562 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309973 2580 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 10 12:18:00.314562 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309976 2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 10 12:18:00.314562 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309979 2580 flags.go:64] FLAG: --experimental-mounter-path="" Sep 10 12:18:00.314562 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309982 2580 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 10 12:18:00.314562 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309985 2580 flags.go:64] FLAG: --fail-swap-on="true" Sep 10 12:18:00.314562 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309988 2580 flags.go:64] FLAG: --feature-gates="" Sep 10 12:18:00.314562 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309992 2580 flags.go:64] FLAG: --file-check-frequency="20s" Sep 10 12:18:00.314562 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309995 2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 10 12:18:00.314562 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.309999 2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 10 12:18:00.314562 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310003 2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 10 12:18:00.314562 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310006 2580 flags.go:64] FLAG: --healthz-port="10248" Sep 10 12:18:00.314562 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310009 2580 flags.go:64] FLAG: --help="false" Sep 10 12:18:00.314562 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310012 2580 flags.go:64] FLAG: --hostname-override="ip-10-0-130-221.ec2.internal" Sep 10 12:18:00.314562 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310015 2580 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 10 12:18:00.315235 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310019 2580 flags.go:64] FLAG: --http-check-frequency="20s" Sep 10 12:18:00.315235 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310022 2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 10 12:18:00.315235 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310025 2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 10 12:18:00.315235 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310029 2580 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 10 12:18:00.315235 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310032 2580 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 10 12:18:00.315235 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310035 2580 flags.go:64] FLAG: --image-service-endpoint="" Sep 10 12:18:00.315235 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310038 2580 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 10 12:18:00.315235 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310043 2580 flags.go:64] FLAG: --kube-api-burst="100" Sep 10 12:18:00.315235 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310047 2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 10 12:18:00.315235 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310050 2580 flags.go:64] FLAG: --kube-api-qps="50" Sep 10 12:18:00.315235 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310054 2580 flags.go:64] FLAG: --kube-reserved="" Sep 10 12:18:00.315235 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310056 2580 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 10 12:18:00.315235 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310059 2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 10 12:18:00.315235 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310063 2580 flags.go:64] FLAG: --kubelet-cgroups="" Sep 10 12:18:00.315235 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310066 2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 10 12:18:00.315235 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310069 2580 flags.go:64] FLAG: --lock-file="" Sep 10 12:18:00.315235 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310072 2580 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 10 12:18:00.315235 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310076 2580 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 10 12:18:00.315235 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310079 2580 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 10 12:18:00.315235 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310084 2580 flags.go:64] FLAG: --log-json-split-stream="false" Sep 10 12:18:00.315235 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310087 2580 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 10 12:18:00.315235 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310090 2580 flags.go:64] FLAG: --log-text-split-stream="false" Sep 10 12:18:00.315235 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310093 2580 flags.go:64] FLAG: --logging-format="text" Sep 10 12:18:00.315810 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310097 2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 10 12:18:00.315810 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310100 2580 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 10 12:18:00.315810 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310103 2580 flags.go:64] FLAG: --manifest-url="" Sep 10 12:18:00.315810 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310106 2580 flags.go:64] FLAG: --manifest-url-header="" Sep 10 12:18:00.315810 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310111 2580 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 10 12:18:00.315810 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310115 2580 flags.go:64] FLAG: --max-open-files="1000000" Sep 10 12:18:00.315810 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310119 2580 flags.go:64] FLAG: --max-pods="110" Sep 10 12:18:00.315810 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310122 2580 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 10 12:18:00.315810 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310125 2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 10 12:18:00.315810 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310128 2580 flags.go:64] FLAG: --memory-manager-policy="None" Sep 10 12:18:00.315810 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310131 2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 10 12:18:00.315810 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310134 2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 10 12:18:00.315810 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310137 2580 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 10 12:18:00.315810 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310140 2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 10 12:18:00.315810 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310149 2580 flags.go:64] FLAG: --node-status-max-images="50" Sep 10 12:18:00.315810 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310152 2580 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 10 12:18:00.315810 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310156 2580 flags.go:64] FLAG: --oom-score-adj="-999" Sep 10 12:18:00.315810 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310159 2580 flags.go:64] FLAG: --pod-cidr="" Sep 10 12:18:00.315810 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310163 2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 10 12:18:00.315810 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310170 2580 flags.go:64] FLAG: --pod-manifest-path="" Sep 10 12:18:00.315810 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310173 2580 flags.go:64] FLAG: --pod-max-pids="-1" Sep 10 12:18:00.315810 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310176 2580 flags.go:64] FLAG: --pods-per-core="0" Sep 10 12:18:00.315810 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310179 2580 flags.go:64] FLAG: --port="10250" Sep 10 12:18:00.315810 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310182 2580 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 10 12:18:00.316479 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310185 2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-087f3b55a2e78b80e" Sep 10 12:18:00.316479 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310188 2580 flags.go:64] FLAG: --qos-reserved="" Sep 10 12:18:00.316479 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310191 2580 flags.go:64] FLAG: --read-only-port="10255" Sep 10 12:18:00.316479 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310194 2580 flags.go:64] FLAG: --register-node="true" Sep 10 12:18:00.316479 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310197 2580 flags.go:64] FLAG: --register-with-taints="" Sep 10 12:18:00.316479 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310201 2580 flags.go:64] FLAG: --registry-burst="10" Sep 10 12:18:00.316479 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310204 2580 flags.go:64] FLAG: --registry-qps="5" Sep 10 12:18:00.316479 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310208 2580 flags.go:64] FLAG: --reserved-cpus="" Sep 10 12:18:00.316479 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310211 2580 flags.go:64] FLAG: --reserved-memory="" Sep 10 12:18:00.316479 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310215 2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 10 12:18:00.316479 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310218 2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 10 12:18:00.316479 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310224 2580 flags.go:64] FLAG: --rotate-certificates="false" Sep 10 12:18:00.316479 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310227 2580 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 10 12:18:00.316479 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310230 2580 flags.go:64] FLAG: --runonce="false" Sep 10 12:18:00.316479 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310233 2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 10 12:18:00.316479 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310236 2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 10 12:18:00.316479 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310239 2580 flags.go:64] FLAG: --seccomp-default="false" Sep 10 12:18:00.316479 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310243 2580 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 10 12:18:00.316479 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310246 2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 10 12:18:00.316479 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310249 2580 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 10 12:18:00.316479 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310252 2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 10 12:18:00.316479 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310255 2580 flags.go:64] FLAG: --storage-driver-password="root" Sep 10 12:18:00.316479 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310258 2580 flags.go:64] FLAG: --storage-driver-secure="false" Sep 10 12:18:00.316479 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310261 2580 flags.go:64] FLAG: --storage-driver-table="stats" Sep 10 12:18:00.316479 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310267 2580 flags.go:64] FLAG: --storage-driver-user="root" Sep 10 12:18:00.317112 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310270 2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 10 12:18:00.317112 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310273 2580 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 10 12:18:00.317112 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310277 2580 flags.go:64] FLAG: --system-cgroups="" Sep 10 12:18:00.317112 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310280 2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 10 12:18:00.317112 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310286 2580 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 10 12:18:00.317112 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310289 2580 flags.go:64] FLAG: --tls-cert-file="" Sep 10 12:18:00.317112 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310292 2580 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 10 12:18:00.317112 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310296 2580 flags.go:64] FLAG: --tls-min-version="" Sep 10 12:18:00.317112 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310299 2580 flags.go:64] FLAG: --tls-private-key-file="" Sep 10 12:18:00.317112 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310302 2580 flags.go:64] FLAG: --topology-manager-policy="none" Sep 10 12:18:00.317112 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310305 2580 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 10 12:18:00.317112 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310307 2580 flags.go:64] FLAG: --topology-manager-scope="container" Sep 10 12:18:00.317112 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310311 2580 flags.go:64] FLAG: --v="2" Sep 10 12:18:00.317112 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310314 2580 flags.go:64] FLAG: --version="false" Sep 10 12:18:00.317112 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310319 2580 flags.go:64] FLAG: --vmodule="" Sep 10 12:18:00.317112 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310322 2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 10 12:18:00.317112 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.310326 2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 10 12:18:00.317112 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310453 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 12:18:00.317112 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310460 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 12:18:00.317112 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310463 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 12:18:00.317112 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310466 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 12:18:00.317112 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310470 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 12:18:00.317112 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310473 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 12:18:00.317680 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310476 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 12:18:00.317680 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310478 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 12:18:00.317680 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310481 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 12:18:00.317680 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310484 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 12:18:00.317680 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310486 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 12:18:00.317680 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310489 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 12:18:00.317680 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310492 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 12:18:00.317680 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310494 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 12:18:00.317680 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310498 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 12:18:00.317680 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310501 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 12:18:00.317680 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310505 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 12:18:00.317680 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310508 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 12:18:00.317680 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310511 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 12:18:00.317680 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310514 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 12:18:00.317680 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310517 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 12:18:00.317680 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310519 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 12:18:00.317680 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310522 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 12:18:00.317680 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310525 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 12:18:00.317680 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310527 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 12:18:00.318163 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310530 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 12:18:00.318163 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310532 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 12:18:00.318163 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310535 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 12:18:00.318163 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310537 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 12:18:00.318163 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310540 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 12:18:00.318163 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310543 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 12:18:00.318163 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310545 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 12:18:00.318163 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310548 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 12:18:00.318163 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310552 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 12:18:00.318163 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310555 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 12:18:00.318163 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310557 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 12:18:00.318163 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310560 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 12:18:00.318163 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310562 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 12:18:00.318163 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310565 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 12:18:00.318163 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310568 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 12:18:00.318163 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310570 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 12:18:00.318163 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310573 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 12:18:00.318163 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310575 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 12:18:00.318163 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310578 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 12:18:00.318163 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310581 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 12:18:00.318666 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310585 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 12:18:00.318666 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310591 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 12:18:00.318666 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310594 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 12:18:00.318666 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310597 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 12:18:00.318666 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310600 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 12:18:00.318666 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310603 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 12:18:00.318666 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310606 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 12:18:00.318666 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310609 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 12:18:00.318666 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310612 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 12:18:00.318666 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310614 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 12:18:00.318666 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310617 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 12:18:00.318666 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310620 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 12:18:00.318666 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310622 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 12:18:00.318666 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310625 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 10 12:18:00.318666 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310628 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 12:18:00.318666 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310631 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 12:18:00.318666 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310633 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 12:18:00.318666 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310636 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 12:18:00.318666 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310639 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 12:18:00.318666 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310642 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 12:18:00.319182 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310657 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 12:18:00.319182 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310659 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 12:18:00.319182 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310664 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 12:18:00.319182 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310668 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 12:18:00.319182 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310671 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 12:18:00.319182 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310674 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 12:18:00.319182 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310678 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 12:18:00.319182 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310681 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 12:18:00.319182 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310683 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 12:18:00.319182 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310686 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 12:18:00.319182 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310689 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 12:18:00.319182 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310691 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 12:18:00.319182 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310694 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 12:18:00.319182 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310698 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 12:18:00.319182 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310701 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 12:18:00.319182 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310704 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 12:18:00.319182 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310707 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 12:18:00.319182 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310709 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 12:18:00.319182 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310714 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 12:18:00.319661 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310718 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 12:18:00.319661 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310721 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 12:18:00.319661 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310724 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 12:18:00.319661 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310727 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 12:18:00.319661 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310730 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 12:18:00.319661 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310733 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 12:18:00.319661 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310736 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 12:18:00.319661 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310739 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 12:18:00.319661 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310741 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 12:18:00.319661 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310744 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 12:18:00.319661 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310781 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 12:18:00.319661 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310801 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 12:18:00.319661 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310806 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 12:18:00.319661 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310810 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 12:18:00.319661 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.310815 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 12:18:00.320138 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.311486 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 12:18:00.320138 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.320012 2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 10 12:18:00.320138 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.320034 2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 10 12:18:00.320895 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.320882 2580 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 10 12:18:00.320936 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.320895 2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 10 12:18:00.320984 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.320974 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 12:18:00.320984 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.320983 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 12:18:00.321039 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.320986 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 12:18:00.321039 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.320990 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 12:18:00.321039 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.320993 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 12:18:00.321039 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.320996 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 12:18:00.321039 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321000 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 12:18:00.321039 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321005 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 12:18:00.321039 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321009 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 12:18:00.321039 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321012 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 12:18:00.321039 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321016 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 12:18:00.321039 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321018 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 12:18:00.321039 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321021 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 12:18:00.321039 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321025 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 12:18:00.321039 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321029 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 12:18:00.321039 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321032 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 12:18:00.321039 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321035 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 12:18:00.321039 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321038 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 12:18:00.321039 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321041 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 12:18:00.321039 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321043 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 12:18:00.321039 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321046 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 12:18:00.321509 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321050 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 10 12:18:00.321509 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321060 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 12:18:00.321509 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321063 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 12:18:00.321509 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321065 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 12:18:00.321509 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321068 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 12:18:00.321509 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321071 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 12:18:00.321509 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321074 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 12:18:00.321509 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321076 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 12:18:00.321509 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321078 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 12:18:00.321509 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321081 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 12:18:00.321509 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321084 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 12:18:00.321509 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321086 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 12:18:00.321509 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321089 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 12:18:00.321509 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321091 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 12:18:00.321509 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321094 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 12:18:00.321509 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321096 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 12:18:00.321509 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321098 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 12:18:00.321509 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321101 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 12:18:00.321509 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321104 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 12:18:00.321509 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321106 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 12:18:00.322097 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321109 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 12:18:00.322097 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321111 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 12:18:00.322097 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321114 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 12:18:00.322097 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321116 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 12:18:00.322097 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321119 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 12:18:00.322097 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321121 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 12:18:00.322097 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321124 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 12:18:00.322097 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321126 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 12:18:00.322097 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321129 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 12:18:00.322097 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321131 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 12:18:00.322097 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321133 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 12:18:00.322097 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321136 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 12:18:00.322097 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321139 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 12:18:00.322097 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321143 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 12:18:00.322097 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321152 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 12:18:00.322097 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321155 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 12:18:00.322097 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321158 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 12:18:00.322097 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321161 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 12:18:00.322097 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321163 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 12:18:00.322097 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321166 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 12:18:00.322591 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321168 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 12:18:00.322591 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321171 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 12:18:00.322591 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321174 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 12:18:00.322591 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321176 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 12:18:00.322591 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321179 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 12:18:00.322591 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321181 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 12:18:00.322591 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321184 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 12:18:00.322591 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321186 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 12:18:00.322591 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321189 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 12:18:00.322591 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321192 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 12:18:00.322591 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321194 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 12:18:00.322591 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321197 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 12:18:00.322591 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321199 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 12:18:00.322591 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321202 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 12:18:00.322591 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321204 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 12:18:00.322591 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321207 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 12:18:00.322591 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321210 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 12:18:00.322591 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321212 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 12:18:00.322591 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321215 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 12:18:00.322591 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321218 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 12:18:00.323112 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321221 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 12:18:00.323112 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321223 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 12:18:00.323112 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321226 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 12:18:00.323112 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321228 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 12:18:00.323112 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321230 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 12:18:00.323112 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321234 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 12:18:00.323112 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321237 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 12:18:00.323112 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321247 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 12:18:00.323112 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321250 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 12:18:00.323112 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321252 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 12:18:00.323112 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321255 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 12:18:00.323112 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321257 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 12:18:00.323112 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321261 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 12:18:00.323112 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321265 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 12:18:00.323112 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321268 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 12:18:00.323112 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321271 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 12:18:00.323112 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321273 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 12:18:00.323112 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321275 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 12:18:00.323557 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.321281 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 12:18:00.323557 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321443 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 12:18:00.323557 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321448 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 12:18:00.323557 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321451 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 12:18:00.323557 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321454 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 12:18:00.323557 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321457 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 12:18:00.323557 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321460 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 12:18:00.323557 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321462 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 12:18:00.323557 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321465 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 12:18:00.323557 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321468 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 12:18:00.323557 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321471 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 12:18:00.323557 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321474 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 12:18:00.323557 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321477 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 12:18:00.323557 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321479 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 12:18:00.323557 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321482 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 12:18:00.324155 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321485 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 12:18:00.324155 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321488 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 12:18:00.324155 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321490 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 12:18:00.324155 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321493 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 12:18:00.324155 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321496 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 12:18:00.324155 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321499 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 12:18:00.324155 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321501 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 12:18:00.324155 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321510 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 12:18:00.324155 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321513 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 12:18:00.324155 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321515 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 12:18:00.324155 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321518 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 12:18:00.324155 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321521 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 12:18:00.324155 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321524 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 12:18:00.324155 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321526 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 12:18:00.324155 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321529 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 12:18:00.324155 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321532 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 12:18:00.324155 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321534 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 12:18:00.324155 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321537 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 12:18:00.324155 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321540 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 12:18:00.324155 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321542 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 12:18:00.324645 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321545 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 12:18:00.324645 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321549 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 12:18:00.324645 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321554 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 12:18:00.324645 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321558 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 12:18:00.324645 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321562 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 12:18:00.324645 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321567 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 12:18:00.324645 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321571 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 12:18:00.324645 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321575 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 12:18:00.324645 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321578 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 12:18:00.324645 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321581 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 12:18:00.324645 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321583 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 12:18:00.324645 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321586 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 12:18:00.324645 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321589 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 12:18:00.324645 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321592 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 12:18:00.324645 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321595 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 12:18:00.324645 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321598 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 12:18:00.324645 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321601 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 12:18:00.324645 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321604 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 12:18:00.324645 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321607 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 12:18:00.325120 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321610 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 12:18:00.325120 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321620 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 12:18:00.325120 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321623 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 12:18:00.325120 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321625 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 12:18:00.325120 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321628 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 12:18:00.325120 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321630 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 12:18:00.325120 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321633 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 12:18:00.325120 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321635 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 12:18:00.325120 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321638 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 12:18:00.325120 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321641 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 12:18:00.325120 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321643 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 12:18:00.325120 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321646 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 12:18:00.325120 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321649 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 12:18:00.325120 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321651 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 12:18:00.325120 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321654 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 12:18:00.325120 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321657 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 12:18:00.325120 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321659 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 12:18:00.325120 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321662 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 12:18:00.325120 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321665 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 12:18:00.325120 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321667 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 12:18:00.325621 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321670 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 12:18:00.325621 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321673 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 12:18:00.325621 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321675 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 10 12:18:00.325621 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321677 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 12:18:00.325621 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321680 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 12:18:00.325621 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321683 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 12:18:00.325621 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321685 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 12:18:00.325621 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321688 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 12:18:00.325621 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321691 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 12:18:00.325621 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321694 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 12:18:00.325621 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321696 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 12:18:00.325621 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321699 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 12:18:00.325621 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321702 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 12:18:00.325621 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321704 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 12:18:00.325621 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321718 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 12:18:00.325621 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321721 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 12:18:00.325621 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321723 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 12:18:00.325621 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321726 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 12:18:00.325621 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321728 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 12:18:00.325621 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321731 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 12:18:00.326112 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321734 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 12:18:00.326112 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321736 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 12:18:00.326112 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321739 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 12:18:00.326112 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321741 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 12:18:00.326112 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321744 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 12:18:00.326112 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:00.321746 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 12:18:00.326112 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.321752 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 12:18:00.326112 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.321935 2580 server.go:962] "Client rotation is on, will bootstrap in background" Sep 10 12:18:00.326112 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.324055 2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 10 12:18:00.326112 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.325208 2580 server.go:1019] "Starting client certificate rotation" Sep 10 12:18:00.326112 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.325310 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 12:18:00.326112 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.325349 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 12:18:00.329123 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.329086 2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 10 12:18:00.334095 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.334057 2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 10 12:18:00.345896 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.345867 2580 log.go:25] "Validated CRI v1 runtime API" Sep 10 12:18:00.349745 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.349723 2580 log.go:25] "Validated CRI v1 image API" Sep 10 12:18:00.351751 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.351730 2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 10 12:18:00.355596 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.355570 2580 fs.go:135] Filesystem UUIDs: map[0e69fc5c-1487-4d5c-8175-835a73248419:/dev/nvme0n1p4 709a74b9-e748-4945-aae4-93e69bdd5a0f:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Sep 10 12:18:00.355707 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.355595 2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 10 12:18:00.359667 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.359641 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 10 12:18:00.361468 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.361337 2580 manager.go:216] Machine: {Timestamp:2026-09-10 12:18:00.359827896 +0000 UTC m=+0.367144271 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097789 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2cddb4fe9564b8e0ff6dbee6c3b81a SystemUUID:ec2cddb4-fe95-64b8-e0ff-6dbee6c3b81a BootID:5ba18ba3-460b-4ef7-9cf1-fd1a47853f03 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:9a:cb:65:8e:2b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:9a:cb:65:8e:2b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:96:9e:da:dc:de:9e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 10 12:18:00.361468 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.361460 2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 10 12:18:00.361655 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.361596 2580 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 10 12:18:00.363309 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.363278 2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 10 12:18:00.363524 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.363314 2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-221.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 10 12:18:00.364650 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.364636 2580 topology_manager.go:138] "Creating topology manager with none policy" Sep 10 12:18:00.364686 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.364654 2580 container_manager_linux.go:309] "Creating device plugin manager" Sep 10 12:18:00.364686 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.364666 2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 10 12:18:00.365499 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.365484 2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 10 12:18:00.366725 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.366710 2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 10 12:18:00.371308 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.371287 2580 state_mem.go:36] "Initialized new in-memory state store" Sep 10 12:18:00.371497 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.371485 2580 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 10 12:18:00.371543 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.371528 2580 kubelet.go:486] "Attempting to sync node with API server" Sep 10 12:18:00.371575 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.371548 2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 10 12:18:00.371575 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.371565 2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 10 12:18:00.371641 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.371580 2580 kubelet.go:398] "Adding apiserver pod source" Sep 10 12:18:00.371641 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.371596 2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 10 12:18:00.372188 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.372173 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 10 12:18:00.372272 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.372244 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 10 12:18:00.374808 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.374784 2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 10 12:18:00.376952 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.376935 2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 10 12:18:00.377026 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.376967 2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 10 12:18:00.377146 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.377136 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 10 12:18:00.377189 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.377152 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 10 12:18:00.377189 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.377158 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 10 12:18:00.377189 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.377166 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 10 12:18:00.377189 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.377173 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 10 12:18:00.377189 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.377179 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 10 12:18:00.377584 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.377576 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 10 12:18:00.377618 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.377585 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 10 12:18:00.377618 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.377593 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 10 12:18:00.377618 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.377599 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 10 12:18:00.377618 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.377609 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 10 12:18:00.377618 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.377616 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 10 12:18:00.378153 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.378143 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 10 12:18:00.378153 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.378153 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 10 12:18:00.382331 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.382305 2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-221.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 12:18:00.382331 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:00.382299 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-221.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 10 12:18:00.382490 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:00.382309 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 10 12:18:00.383540 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.383526 2580 server.go:1268] "Started kubelet" Sep 10 12:18:00.383694 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.383658 2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 10 12:18:00.383809 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.383652 2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 10 12:18:00.383809 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.383741 2580 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 10 12:18:00.384732 ip-10-0-130-221 systemd[1]: Started Kubernetes Kubelet. Sep 10 12:18:00.385292 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.385274 2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 10 12:18:00.385932 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.385815 2580 server.go:310] "Adding debug handlers to kubelet server" Sep 10 12:18:00.388843 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:00.388787 2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 10 12:18:00.389011 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.388894 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 10 12:18:00.389011 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.388915 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-k9gsh" Sep 10 12:18:00.389284 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.389267 2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 10 12:18:00.389532 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:00.389513 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-221.ec2.internal\" not found" Sep 10 12:18:00.389894 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.389880 2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 10 12:18:00.389984 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.389906 2580 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 10 12:18:00.389984 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.389918 2580 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 10 12:18:00.390153 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.390138 2580 reconstruct.go:97] "Volume reconstruction finished" Sep 10 12:18:00.390208 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.390174 2580 reconciler.go:29] "Reconciler: start to sync state" Sep 10 12:18:00.390275 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.390260 2580 factory.go:55] Registering systemd factory Sep 10 12:18:00.390325 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.390279 2580 factory.go:223] Registration of the systemd container factory successfully Sep 10 12:18:00.390555 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.390541 2580 factory.go:153] Registering CRI-O factory Sep 10 12:18:00.390622 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.390560 2580 factory.go:223] Registration of the crio container factory successfully Sep 10 12:18:00.390622 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.390618 2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 10 12:18:00.390713 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.390641 2580 factory.go:103] Registering Raw factory Sep 10 12:18:00.390713 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.390657 2580 manager.go:1196] Started watching for new ooms in manager Sep 10 12:18:00.391024 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.391011 2580 manager.go:349] Starting recovery of all containers Sep 10 12:18:00.392526 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:00.392494 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 10 12:18:00.392619 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:00.392592 2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-221.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 10 12:18:00.393567 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:00.392283 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-221.ec2.internal.18d3f4d6cc453886 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-221.ec2.internal,UID:ip-10-0-130-221.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-221.ec2.internal,},FirstTimestamp:2026-09-10 12:18:00.383486086 +0000 UTC m=+0.390802460,LastTimestamp:2026-09-10 12:18:00.383486086 +0000 UTC m=+0.390802460,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-221.ec2.internal,}" Sep 10 12:18:00.397358 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.397330 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-k9gsh" Sep 10 12:18:00.404123 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.404074 2580 manager.go:354] Recovery completed Sep 10 12:18:00.410240 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.410223 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 12:18:00.412449 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.412427 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-221.ec2.internal" event="NodeHasSufficientMemory" Sep 10 12:18:00.412549 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.412463 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-221.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 12:18:00.412549 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.412477 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-221.ec2.internal" event="NodeHasSufficientPID" Sep 10 12:18:00.413125 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.413111 2580 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 10 12:18:00.413125 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.413118 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 12:18:00.413125 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.413123 2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 10 12:18:00.413251 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.413141 2580 state_mem.go:36] "Initialized new in-memory state store" Sep 10 12:18:00.413918 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.413900 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-221.ec2.internal" event="NodeHasSufficientMemory" Sep 10 12:18:00.413983 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.413936 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-221.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 12:18:00.413983 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.413951 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-221.ec2.internal" event="NodeHasSufficientPID" Sep 10 12:18:00.414684 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:00.414599 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-221.ec2.internal.18d3f4d6cdff1f01 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-221.ec2.internal,UID:ip-10-0-130-221.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-221.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-221.ec2.internal,},FirstTimestamp:2026-09-10 12:18:00.412446465 +0000 UTC m=+0.419762838,LastTimestamp:2026-09-10 12:18:00.412446465 +0000 UTC m=+0.419762838,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-221.ec2.internal,}" Sep 10 12:18:00.415394 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.415355 2580 policy_none.go:49] "None policy: Start" Sep 10 12:18:00.415394 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.415385 2580 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 10 12:18:00.415562 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.415399 2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 10 12:18:00.417563 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.417549 2580 policy_none.go:47] "Start" Sep 10 12:18:00.446587 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.446563 2580 manager.go:337] "Starting Device Plugin manager" Sep 10 12:18:00.473166 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:00.446629 2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 10 12:18:00.473166 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.446640 2580 server.go:87] "Starting device plugin registration server" Sep 10 12:18:00.473166 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.447064 2580 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 10 12:18:00.473166 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.447079 2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 10 12:18:00.473166 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.447176 2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 10 12:18:00.473166 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.447271 2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 10 12:18:00.473166 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.447280 2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 10 12:18:00.473166 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:00.447931 2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 10 12:18:00.473166 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:00.447977 2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-221.ec2.internal\" not found" Sep 10 12:18:00.547880 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.547827 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 12:18:00.549151 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.549133 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-221.ec2.internal" event="NodeHasSufficientMemory" Sep 10 12:18:00.549261 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.549166 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-221.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 12:18:00.549261 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.549177 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-221.ec2.internal" event="NodeHasSufficientPID" Sep 10 12:18:00.549261 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.549207 2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-221.ec2.internal" Sep 10 12:18:00.549976 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.549901 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 10 12:18:00.551132 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.551111 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 10 12:18:00.551230 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.551166 2580 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 10 12:18:00.551230 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.551191 2580 kubelet.go:2448] "Starting kubelet main sync loop" Sep 10 12:18:00.551230 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:00.551232 2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 10 12:18:00.553315 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.553286 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 12:18:00.558329 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.558308 2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-221.ec2.internal" Sep 10 12:18:00.558408 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:00.558337 2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-221.ec2.internal\": node \"ip-10-0-130-221.ec2.internal\" not found" Sep 10 12:18:00.619959 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:00.619921 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-221.ec2.internal\" not found" Sep 10 12:18:00.652042 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.651992 2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-221.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-221.ec2.internal"] Sep 10 12:18:00.652204 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.652120 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 12:18:00.654159 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.654138 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-221.ec2.internal" event="NodeHasSufficientMemory" Sep 10 12:18:00.654282 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.654173 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-221.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 12:18:00.654282 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.654184 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-221.ec2.internal" event="NodeHasSufficientPID" Sep 10 12:18:00.656253 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.656240 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 12:18:00.656396 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.656378 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-221.ec2.internal" Sep 10 12:18:00.656432 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.656420 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 12:18:00.657175 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.657160 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-221.ec2.internal" event="NodeHasSufficientMemory" Sep 10 12:18:00.657246 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.657188 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-221.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 12:18:00.657246 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.657198 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-221.ec2.internal" event="NodeHasSufficientPID" Sep 10 12:18:00.657246 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.657164 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-221.ec2.internal" event="NodeHasSufficientMemory" Sep 10 12:18:00.657343 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.657262 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-221.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 12:18:00.657343 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.657273 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-221.ec2.internal" event="NodeHasSufficientPID" Sep 10 12:18:00.659198 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.659184 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-221.ec2.internal" Sep 10 12:18:00.659260 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.659212 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 12:18:00.661065 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.661046 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-221.ec2.internal" event="NodeHasSufficientMemory" Sep 10 12:18:00.661154 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.661081 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-221.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 12:18:00.661154 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.661097 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-221.ec2.internal" event="NodeHasSufficientPID" Sep 10 12:18:00.675643 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:00.675613 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-221.ec2.internal\" not found" node="ip-10-0-130-221.ec2.internal" Sep 10 12:18:00.680138 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:00.680101 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-221.ec2.internal\" not found" node="ip-10-0-130-221.ec2.internal" Sep 10 12:18:00.691895 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.691845 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b4983c0fc993a6c067676ddbfae0d2d6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-221.ec2.internal\" (UID: \"b4983c0fc993a6c067676ddbfae0d2d6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-221.ec2.internal" Sep 10 12:18:00.692018 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.691899 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b4983c0fc993a6c067676ddbfae0d2d6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-221.ec2.internal\" (UID: \"b4983c0fc993a6c067676ddbfae0d2d6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-221.ec2.internal" Sep 10 12:18:00.692018 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.691930 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dbeff53fae53b2340a3f2a4afbf5b2aa-config\") pod \"kube-apiserver-proxy-ip-10-0-130-221.ec2.internal\" (UID: \"dbeff53fae53b2340a3f2a4afbf5b2aa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-221.ec2.internal" Sep 10 12:18:00.720663 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:00.720631 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-221.ec2.internal\" not found" Sep 10 12:18:00.792696 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.792661 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b4983c0fc993a6c067676ddbfae0d2d6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-221.ec2.internal\" (UID: \"b4983c0fc993a6c067676ddbfae0d2d6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-221.ec2.internal" Sep 10 12:18:00.792815 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.792715 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b4983c0fc993a6c067676ddbfae0d2d6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-221.ec2.internal\" (UID: \"b4983c0fc993a6c067676ddbfae0d2d6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-221.ec2.internal" Sep 10 12:18:00.792815 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.792737 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dbeff53fae53b2340a3f2a4afbf5b2aa-config\") pod \"kube-apiserver-proxy-ip-10-0-130-221.ec2.internal\" (UID: \"dbeff53fae53b2340a3f2a4afbf5b2aa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-221.ec2.internal" Sep 10 12:18:00.792815 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.792664 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b4983c0fc993a6c067676ddbfae0d2d6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-221.ec2.internal\" (UID: \"b4983c0fc993a6c067676ddbfae0d2d6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-221.ec2.internal" Sep 10 12:18:00.792815 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.792789 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dbeff53fae53b2340a3f2a4afbf5b2aa-config\") pod \"kube-apiserver-proxy-ip-10-0-130-221.ec2.internal\" (UID: \"dbeff53fae53b2340a3f2a4afbf5b2aa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-221.ec2.internal" Sep 10 12:18:00.792815 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.792799 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b4983c0fc993a6c067676ddbfae0d2d6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-221.ec2.internal\" (UID: \"b4983c0fc993a6c067676ddbfae0d2d6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-221.ec2.internal" Sep 10 12:18:00.821846 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:00.821754 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-221.ec2.internal\" not found" Sep 10 12:18:00.922614 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:00.922563 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-221.ec2.internal\" not found" Sep 10 12:18:00.978777 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.978744 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-221.ec2.internal" Sep 10 12:18:00.982339 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:00.982320 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-221.ec2.internal" Sep 10 12:18:01.023575 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:01.023529 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-221.ec2.internal\" not found" Sep 10 12:18:01.124284 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:01.124171 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-221.ec2.internal\" not found" Sep 10 12:18:01.224726 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:01.224688 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-221.ec2.internal\" not found" Sep 10 12:18:01.325818 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:01.325378 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-221.ec2.internal\" not found" Sep 10 12:18:01.325818 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.325445 2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 10 12:18:01.325818 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.325620 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 10 12:18:01.359376 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.359343 2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 12:18:01.372765 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.372737 2580 apiserver.go:52] "Watching apiserver" Sep 10 12:18:01.386764 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.386690 2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 10 12:18:01.387641 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.387618 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-ljkj8","kube-system/konnectivity-agent-zjw9d","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5mmp2","openshift-cluster-node-tuning-operator/tuned-t8ths","openshift-dns/node-resolver-pkt25","openshift-image-registry/node-ca-68mrh","openshift-multus/network-metrics-daemon-7vx2g","openshift-network-operator/iptables-alerter-ktr9w","openshift-multus/multus-5g7lk","openshift-multus/multus-additional-cni-plugins-ft7lg","openshift-network-diagnostics/network-check-target-d7ls5","openshift-ovn-kubernetes/ovnkube-node-tfcmz"] Sep 10 12:18:01.389306 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.389287 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 10 12:18:01.389873 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.389790 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-221.ec2.internal" Sep 10 12:18:01.390736 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.390706 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:01.390886 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:01.390828 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkj8" podUID="67095b3f-a8e2-494b-9f83-b4eb0b5634a2" Sep 10 12:18:01.392714 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.392686 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zjw9d" Sep 10 12:18:01.394878 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.394843 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5mmp2" Sep 10 12:18:01.395534 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.395513 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 10 12:18:01.395619 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.395557 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 10 12:18:01.395762 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.395750 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-j7smm\"" Sep 10 12:18:01.396317 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.396298 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/67095b3f-a8e2-494b-9f83-b4eb0b5634a2-kubelet-config\") pod \"global-pull-secret-syncer-ljkj8\" (UID: \"67095b3f-a8e2-494b-9f83-b4eb0b5634a2\") " pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:01.396391 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.396329 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/67095b3f-a8e2-494b-9f83-b4eb0b5634a2-dbus\") pod \"global-pull-secret-syncer-ljkj8\" (UID: \"67095b3f-a8e2-494b-9f83-b4eb0b5634a2\") " pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:01.396391 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.396355 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/67095b3f-a8e2-494b-9f83-b4eb0b5634a2-original-pull-secret\") pod \"global-pull-secret-syncer-ljkj8\" (UID: \"67095b3f-a8e2-494b-9f83-b4eb0b5634a2\") " pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:01.396945 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.396931 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.398186 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.398168 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 10 12:18:01.398186 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.398177 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 10 12:18:01.398327 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.398178 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-xqhmd\"" Sep 10 12:18:01.398786 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.398770 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 10 12:18:01.399090 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.399077 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pkt25" Sep 10 12:18:01.399394 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.399353 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-2zcc8\"" Sep 10 12:18:01.399540 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.399525 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 10 12:18:01.399913 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.399897 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 10 12:18:01.400560 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.400522 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 12:13:00 +0000 UTC" deadline="2028-04-14 23:43:52.532705175 +0000 UTC" Sep 10 12:18:01.400628 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.400559 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13979h25m51.132150076s" Sep 10 12:18:01.401211 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.401197 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-68mrh" Sep 10 12:18:01.401727 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.401711 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 10 12:18:01.401800 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.401785 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 10 12:18:01.402024 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.402011 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-l6jq4\"" Sep 10 12:18:01.403312 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.403300 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:01.403373 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:01.403358 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vx2g" podUID="7169f42e-6be8-4806-bf45-c77b6e2bdfd1" Sep 10 12:18:01.403470 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.403452 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 10 12:18:01.403749 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.403737 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-brs5v\"" Sep 10 12:18:01.403950 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.403936 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 10 12:18:01.404129 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.404115 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 10 12:18:01.405411 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.405390 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ktr9w" Sep 10 12:18:01.405790 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.405772 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 10 12:18:01.405881 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.405838 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-221.ec2.internal" Sep 10 12:18:01.407605 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.407588 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.408253 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.408228 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 10 12:18:01.408377 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.408356 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-ldw8g\"" Sep 10 12:18:01.408502 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.408479 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 10 12:18:01.408750 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.408734 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 10 12:18:01.410098 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.410078 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ft7lg" Sep 10 12:18:01.410191 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.410179 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-h8c5f\"" Sep 10 12:18:01.410242 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.410227 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 10 12:18:01.410513 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.410496 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 10 12:18:01.410783 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.410767 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 10 12:18:01.410882 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.410786 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 10 12:18:01.412371 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.412356 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:18:01.412443 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:01.412426 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d7ls5" podUID="1b89f971-1fee-46bf-ad4c-e68e6b83b9a0" Sep 10 12:18:01.414055 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.414036 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 10 12:18:01.414127 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.414061 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 10 12:18:01.414280 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.414265 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-5bknk\"" Sep 10 12:18:01.414347 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.414303 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 10 12:18:01.414566 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.414549 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-221.ec2.internal"] Sep 10 12:18:01.414644 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.414631 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.417162 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.417138 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-m9z4m\"" Sep 10 12:18:01.417541 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.417523 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 10 12:18:01.417623 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.417589 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 10 12:18:01.417623 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.417614 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 10 12:18:01.417747 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.417734 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 10 12:18:01.417871 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.417844 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 10 12:18:01.417962 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.417943 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 10 12:18:01.421496 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.421476 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 10 12:18:01.421623 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.421601 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-130-221.ec2.internal"] Sep 10 12:18:01.437902 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.437177 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-kcwj4" Sep 10 12:18:01.444221 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.444196 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-kcwj4" Sep 10 12:18:01.490784 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.490764 2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 10 12:18:01.496540 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.496511 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-host-run-multus-certs\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.496696 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.496546 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-host-slash\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.496696 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.496564 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-run-ovn\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.496696 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.496588 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-log-socket\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.496696 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.496622 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-ovnkube-config\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.496696 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.496654 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p79cx\" (UniqueName: \"kubernetes.io/projected/1d17b055-9484-48c5-bd77-90870f87d6ff-kube-api-access-p79cx\") pod \"multus-additional-cni-plugins-ft7lg\" (UID: \"1d17b055-9484-48c5-bd77-90870f87d6ff\") " pod="openshift-multus/multus-additional-cni-plugins-ft7lg" Sep 10 12:18:01.496696 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.496690 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6lf4\" (UniqueName: \"kubernetes.io/projected/e6b25f25-3e5d-45f3-8c46-7884a14b981f-kube-api-access-t6lf4\") pod \"node-ca-68mrh\" (UID: \"e6b25f25-3e5d-45f3-8c46-7884a14b981f\") " pod="openshift-image-registry/node-ca-68mrh" Sep 10 12:18:01.496900 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.496710 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-host-var-lib-cni-multus\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.496900 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.496724 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-host-var-lib-kubelet\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.496900 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.496740 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-multus-daemon-config\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.496900 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.496769 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1d17b055-9484-48c5-bd77-90870f87d6ff-cni-binary-copy\") pod \"multus-additional-cni-plugins-ft7lg\" (UID: \"1d17b055-9484-48c5-bd77-90870f87d6ff\") " pod="openshift-multus/multus-additional-cni-plugins-ft7lg" Sep 10 12:18:01.496900 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.496784 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7169f42e-6be8-4806-bf45-c77b6e2bdfd1-metrics-certs\") pod \"network-metrics-daemon-7vx2g\" (UID: \"7169f42e-6be8-4806-bf45-c77b6e2bdfd1\") " pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:01.496900 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.496801 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-host-run-k8s-cni-cncf-io\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.496900 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.496816 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-hostroot\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.496900 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.496866 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-systemd-units\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.496900 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.496900 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-ovn-node-metrics-cert\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.497141 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.496919 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6cda5c27-4386-49bd-8ad1-eb1e86535408-host-slash\") pod \"iptables-alerter-ktr9w\" (UID: \"6cda5c27-4386-49bd-8ad1-eb1e86535408\") " pod="openshift-network-operator/iptables-alerter-ktr9w" Sep 10 12:18:01.497141 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.496934 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1d17b055-9484-48c5-bd77-90870f87d6ff-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ft7lg\" (UID: \"1d17b055-9484-48c5-bd77-90870f87d6ff\") " pod="openshift-multus/multus-additional-cni-plugins-ft7lg" Sep 10 12:18:01.497141 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.496952 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e6b25f25-3e5d-45f3-8c46-7884a14b981f-serviceca\") pod \"node-ca-68mrh\" (UID: \"e6b25f25-3e5d-45f3-8c46-7884a14b981f\") " pod="openshift-image-registry/node-ca-68mrh" Sep 10 12:18:01.497141 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.496968 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-multus-cni-dir\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.497141 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.496988 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/08a3068f-f5f3-4086-8e21-444a4df81149-registration-dir\") pod \"aws-ebs-csi-driver-node-5mmp2\" (UID: \"08a3068f-f5f3-4086-8e21-444a4df81149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5mmp2" Sep 10 12:18:01.497141 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497003 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c502cf69-0e72-4829-bb53-cd23f5367666-var-lib-kubelet\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.497141 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497023 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c502cf69-0e72-4829-bb53-cd23f5367666-host\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.497141 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497040 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-system-cni-dir\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.497141 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497056 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-var-lib-openvswitch\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.497141 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497074 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-host-kubelet\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.497141 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497098 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/08a3068f-f5f3-4086-8e21-444a4df81149-etc-selinux\") pod \"aws-ebs-csi-driver-node-5mmp2\" (UID: \"08a3068f-f5f3-4086-8e21-444a4df81149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5mmp2" Sep 10 12:18:01.497141 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497130 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-multus-socket-dir-parent\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.497461 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497146 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/08a3068f-f5f3-4086-8e21-444a4df81149-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5mmp2\" (UID: \"08a3068f-f5f3-4086-8e21-444a4df81149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5mmp2" Sep 10 12:18:01.497461 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497179 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/08a3068f-f5f3-4086-8e21-444a4df81149-sys-fs\") pod \"aws-ebs-csi-driver-node-5mmp2\" (UID: \"08a3068f-f5f3-4086-8e21-444a4df81149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5mmp2" Sep 10 12:18:01.497461 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497206 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c502cf69-0e72-4829-bb53-cd23f5367666-sys\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.497461 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497245 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-node-log\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.497461 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497276 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a76158d-6412-48e2-a091-51cc34552ce2-tmp-dir\") pod \"node-resolver-pkt25\" (UID: \"2a76158d-6412-48e2-a091-51cc34552ce2\") " pod="openshift-dns/node-resolver-pkt25" Sep 10 12:18:01.497461 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497297 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-host-run-netns\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.497461 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497322 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-host-run-ovn-kubernetes\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.497461 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497349 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-ovnkube-script-lib\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.497461 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497373 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/67095b3f-a8e2-494b-9f83-b4eb0b5634a2-kubelet-config\") pod \"global-pull-secret-syncer-ljkj8\" (UID: \"67095b3f-a8e2-494b-9f83-b4eb0b5634a2\") " pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:01.497461 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497389 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/67095b3f-a8e2-494b-9f83-b4eb0b5634a2-dbus\") pod \"global-pull-secret-syncer-ljkj8\" (UID: \"67095b3f-a8e2-494b-9f83-b4eb0b5634a2\") " pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:01.497461 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497418 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/67095b3f-a8e2-494b-9f83-b4eb0b5634a2-original-pull-secret\") pod \"global-pull-secret-syncer-ljkj8\" (UID: \"67095b3f-a8e2-494b-9f83-b4eb0b5634a2\") " pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:01.497461 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497437 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5qpj\" (UniqueName: \"kubernetes.io/projected/08a3068f-f5f3-4086-8e21-444a4df81149-kube-api-access-w5qpj\") pod \"aws-ebs-csi-driver-node-5mmp2\" (UID: \"08a3068f-f5f3-4086-8e21-444a4df81149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5mmp2" Sep 10 12:18:01.497461 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497454 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-cnibin\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.498020 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497468 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-run-systemd\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.498020 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497472 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/67095b3f-a8e2-494b-9f83-b4eb0b5634a2-dbus\") pod \"global-pull-secret-syncer-ljkj8\" (UID: \"67095b3f-a8e2-494b-9f83-b4eb0b5634a2\") " pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:01.498020 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497479 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/67095b3f-a8e2-494b-9f83-b4eb0b5634a2-kubelet-config\") pod \"global-pull-secret-syncer-ljkj8\" (UID: \"67095b3f-a8e2-494b-9f83-b4eb0b5634a2\") " pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:01.498020 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497492 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-etc-openvswitch\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.498020 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497524 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3cf0d46b-95a9-4187-9395-ebd8a11b5771-konnectivity-ca\") pod \"konnectivity-agent-zjw9d\" (UID: \"3cf0d46b-95a9-4187-9395-ebd8a11b5771\") " pod="kube-system/konnectivity-agent-zjw9d" Sep 10 12:18:01.498020 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497551 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/08a3068f-f5f3-4086-8e21-444a4df81149-socket-dir\") pod \"aws-ebs-csi-driver-node-5mmp2\" (UID: \"08a3068f-f5f3-4086-8e21-444a4df81149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5mmp2" Sep 10 12:18:01.498020 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:01.497560 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 12:18:01.498020 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497575 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vndkc\" (UniqueName: \"kubernetes.io/projected/6cda5c27-4386-49bd-8ad1-eb1e86535408-kube-api-access-vndkc\") pod \"iptables-alerter-ktr9w\" (UID: \"6cda5c27-4386-49bd-8ad1-eb1e86535408\") " pod="openshift-network-operator/iptables-alerter-ktr9w" Sep 10 12:18:01.498020 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:01.497625 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67095b3f-a8e2-494b-9f83-b4eb0b5634a2-original-pull-secret podName:67095b3f-a8e2-494b-9f83-b4eb0b5634a2 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:01.99760676 +0000 UTC m=+2.004923124 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/67095b3f-a8e2-494b-9f83-b4eb0b5634a2-original-pull-secret") pod "global-pull-secret-syncer-ljkj8" (UID: "67095b3f-a8e2-494b-9f83-b4eb0b5634a2") : object "kube-system"/"original-pull-secret" not registered Sep 10 12:18:01.498020 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497676 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-run-openvswitch\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.498020 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497718 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e6b25f25-3e5d-45f3-8c46-7884a14b981f-host\") pod \"node-ca-68mrh\" (UID: \"e6b25f25-3e5d-45f3-8c46-7884a14b981f\") " pod="openshift-image-registry/node-ca-68mrh" Sep 10 12:18:01.498020 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497745 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-cni-binary-copy\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.498020 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497768 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-multus-conf-dir\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.498020 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497791 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-host-run-netns\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.498020 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497834 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.498020 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497908 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1d17b055-9484-48c5-bd77-90870f87d6ff-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ft7lg\" (UID: \"1d17b055-9484-48c5-bd77-90870f87d6ff\") " pod="openshift-multus/multus-additional-cni-plugins-ft7lg" Sep 10 12:18:01.498664 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497927 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c502cf69-0e72-4829-bb53-cd23f5367666-etc-sysconfig\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.498664 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497944 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c502cf69-0e72-4829-bb53-cd23f5367666-etc-sysctl-d\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.498664 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497960 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c502cf69-0e72-4829-bb53-cd23f5367666-run\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.498664 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497976 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c502cf69-0e72-4829-bb53-cd23f5367666-lib-modules\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.498664 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.497994 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c502cf69-0e72-4829-bb53-cd23f5367666-tmp\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.498664 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.498010 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-host-cni-bin\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.498664 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.498040 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/08a3068f-f5f3-4086-8e21-444a4df81149-device-dir\") pod \"aws-ebs-csi-driver-node-5mmp2\" (UID: \"08a3068f-f5f3-4086-8e21-444a4df81149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5mmp2" Sep 10 12:18:01.498664 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.498069 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c502cf69-0e72-4829-bb53-cd23f5367666-etc-systemd\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.498664 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.498095 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fsrm\" (UniqueName: \"kubernetes.io/projected/2a76158d-6412-48e2-a091-51cc34552ce2-kube-api-access-2fsrm\") pod \"node-resolver-pkt25\" (UID: \"2a76158d-6412-48e2-a091-51cc34552ce2\") " pod="openshift-dns/node-resolver-pkt25" Sep 10 12:18:01.498664 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.498124 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-host-var-lib-cni-bin\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.498664 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.498147 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-host-cni-netd\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.498664 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.498173 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95vgz\" (UniqueName: \"kubernetes.io/projected/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-kube-api-access-95vgz\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.498664 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.498197 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3cf0d46b-95a9-4187-9395-ebd8a11b5771-agent-certs\") pod \"konnectivity-agent-zjw9d\" (UID: \"3cf0d46b-95a9-4187-9395-ebd8a11b5771\") " pod="kube-system/konnectivity-agent-zjw9d" Sep 10 12:18:01.498664 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.498221 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c502cf69-0e72-4829-bb53-cd23f5367666-etc-sysctl-conf\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.498664 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.498242 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c502cf69-0e72-4829-bb53-cd23f5367666-etc-tuned\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.498664 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.498266 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtfw6\" (UniqueName: \"kubernetes.io/projected/c502cf69-0e72-4829-bb53-cd23f5367666-kube-api-access-vtfw6\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.498664 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.498292 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-etc-kubernetes\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.499168 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.498320 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-env-overrides\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.499168 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.498344 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6cda5c27-4386-49bd-8ad1-eb1e86535408-iptables-alerter-script\") pod \"iptables-alerter-ktr9w\" (UID: \"6cda5c27-4386-49bd-8ad1-eb1e86535408\") " pod="openshift-network-operator/iptables-alerter-ktr9w" Sep 10 12:18:01.499168 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.498360 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1d17b055-9484-48c5-bd77-90870f87d6ff-system-cni-dir\") pod \"multus-additional-cni-plugins-ft7lg\" (UID: \"1d17b055-9484-48c5-bd77-90870f87d6ff\") " pod="openshift-multus/multus-additional-cni-plugins-ft7lg" Sep 10 12:18:01.499168 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.498374 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1d17b055-9484-48c5-bd77-90870f87d6ff-cnibin\") pod \"multus-additional-cni-plugins-ft7lg\" (UID: \"1d17b055-9484-48c5-bd77-90870f87d6ff\") " pod="openshift-multus/multus-additional-cni-plugins-ft7lg" Sep 10 12:18:01.499168 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.498395 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c502cf69-0e72-4829-bb53-cd23f5367666-etc-modprobe-d\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.499168 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.498409 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c502cf69-0e72-4829-bb53-cd23f5367666-etc-kubernetes\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.499168 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.498425 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdh47\" (UniqueName: \"kubernetes.io/projected/7169f42e-6be8-4806-bf45-c77b6e2bdfd1-kube-api-access-rdh47\") pod \"network-metrics-daemon-7vx2g\" (UID: \"7169f42e-6be8-4806-bf45-c77b6e2bdfd1\") " pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:01.499168 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.498458 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffmpn\" (UniqueName: \"kubernetes.io/projected/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-kube-api-access-ffmpn\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.499168 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.498478 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-852dl\" (UniqueName: \"kubernetes.io/projected/1b89f971-1fee-46bf-ad4c-e68e6b83b9a0-kube-api-access-852dl\") pod \"network-check-target-d7ls5\" (UID: \"1b89f971-1fee-46bf-ad4c-e68e6b83b9a0\") " pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:18:01.499168 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.498503 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1d17b055-9484-48c5-bd77-90870f87d6ff-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ft7lg\" (UID: \"1d17b055-9484-48c5-bd77-90870f87d6ff\") " pod="openshift-multus/multus-additional-cni-plugins-ft7lg" Sep 10 12:18:01.499168 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.498537 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2a76158d-6412-48e2-a091-51cc34552ce2-hosts-file\") pod \"node-resolver-pkt25\" (UID: \"2a76158d-6412-48e2-a091-51cc34552ce2\") " pod="openshift-dns/node-resolver-pkt25" Sep 10 12:18:01.598887 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.598834 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1d17b055-9484-48c5-bd77-90870f87d6ff-cni-binary-copy\") pod \"multus-additional-cni-plugins-ft7lg\" (UID: \"1d17b055-9484-48c5-bd77-90870f87d6ff\") " pod="openshift-multus/multus-additional-cni-plugins-ft7lg" Sep 10 12:18:01.598887 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.598891 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7169f42e-6be8-4806-bf45-c77b6e2bdfd1-metrics-certs\") pod \"network-metrics-daemon-7vx2g\" (UID: \"7169f42e-6be8-4806-bf45-c77b6e2bdfd1\") " pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:01.599160 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.598909 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-host-run-k8s-cni-cncf-io\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.599160 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.598960 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-host-run-k8s-cni-cncf-io\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.599160 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599010 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-hostroot\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.599160 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:01.599025 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:18:01.599160 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599065 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-systemd-units\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.599160 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599080 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-hostroot\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.599160 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599032 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-systemd-units\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.599160 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:01.599098 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7169f42e-6be8-4806-bf45-c77b6e2bdfd1-metrics-certs podName:7169f42e-6be8-4806-bf45-c77b6e2bdfd1 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:02.099078733 +0000 UTC m=+2.106395097 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7169f42e-6be8-4806-bf45-c77b6e2bdfd1-metrics-certs") pod "network-metrics-daemon-7vx2g" (UID: "7169f42e-6be8-4806-bf45-c77b6e2bdfd1") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:18:01.599511 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599161 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-ovn-node-metrics-cert\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.599511 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599193 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6cda5c27-4386-49bd-8ad1-eb1e86535408-host-slash\") pod \"iptables-alerter-ktr9w\" (UID: \"6cda5c27-4386-49bd-8ad1-eb1e86535408\") " pod="openshift-network-operator/iptables-alerter-ktr9w" Sep 10 12:18:01.599511 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599218 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1d17b055-9484-48c5-bd77-90870f87d6ff-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ft7lg\" (UID: \"1d17b055-9484-48c5-bd77-90870f87d6ff\") " pod="openshift-multus/multus-additional-cni-plugins-ft7lg" Sep 10 12:18:01.599511 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599257 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6cda5c27-4386-49bd-8ad1-eb1e86535408-host-slash\") pod \"iptables-alerter-ktr9w\" (UID: \"6cda5c27-4386-49bd-8ad1-eb1e86535408\") " pod="openshift-network-operator/iptables-alerter-ktr9w" Sep 10 12:18:01.599511 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599293 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e6b25f25-3e5d-45f3-8c46-7884a14b981f-serviceca\") pod \"node-ca-68mrh\" (UID: \"e6b25f25-3e5d-45f3-8c46-7884a14b981f\") " pod="openshift-image-registry/node-ca-68mrh" Sep 10 12:18:01.599511 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599320 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-multus-cni-dir\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.599511 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599347 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/08a3068f-f5f3-4086-8e21-444a4df81149-registration-dir\") pod \"aws-ebs-csi-driver-node-5mmp2\" (UID: \"08a3068f-f5f3-4086-8e21-444a4df81149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5mmp2" Sep 10 12:18:01.599511 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599391 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c502cf69-0e72-4829-bb53-cd23f5367666-var-lib-kubelet\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.599511 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599396 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1d17b055-9484-48c5-bd77-90870f87d6ff-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ft7lg\" (UID: \"1d17b055-9484-48c5-bd77-90870f87d6ff\") " pod="openshift-multus/multus-additional-cni-plugins-ft7lg" Sep 10 12:18:01.599511 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599412 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1d17b055-9484-48c5-bd77-90870f87d6ff-cni-binary-copy\") pod \"multus-additional-cni-plugins-ft7lg\" (UID: \"1d17b055-9484-48c5-bd77-90870f87d6ff\") " pod="openshift-multus/multus-additional-cni-plugins-ft7lg" Sep 10 12:18:01.599511 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599432 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-multus-cni-dir\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.599511 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599460 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/08a3068f-f5f3-4086-8e21-444a4df81149-registration-dir\") pod \"aws-ebs-csi-driver-node-5mmp2\" (UID: \"08a3068f-f5f3-4086-8e21-444a4df81149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5mmp2" Sep 10 12:18:01.599511 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599471 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c502cf69-0e72-4829-bb53-cd23f5367666-var-lib-kubelet\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.599511 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599416 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c502cf69-0e72-4829-bb53-cd23f5367666-host\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.599511 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599505 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-system-cni-dir\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.599511 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599507 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c502cf69-0e72-4829-bb53-cd23f5367666-host\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.600253 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599528 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-var-lib-openvswitch\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.600253 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599550 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-host-kubelet\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.600253 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599576 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/08a3068f-f5f3-4086-8e21-444a4df81149-etc-selinux\") pod \"aws-ebs-csi-driver-node-5mmp2\" (UID: \"08a3068f-f5f3-4086-8e21-444a4df81149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5mmp2" Sep 10 12:18:01.600253 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599599 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-multus-socket-dir-parent\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.600253 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599620 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/08a3068f-f5f3-4086-8e21-444a4df81149-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5mmp2\" (UID: \"08a3068f-f5f3-4086-8e21-444a4df81149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5mmp2" Sep 10 12:18:01.600253 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599637 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-var-lib-openvswitch\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.600253 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599643 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/08a3068f-f5f3-4086-8e21-444a4df81149-sys-fs\") pod \"aws-ebs-csi-driver-node-5mmp2\" (UID: \"08a3068f-f5f3-4086-8e21-444a4df81149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5mmp2" Sep 10 12:18:01.600253 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599666 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/08a3068f-f5f3-4086-8e21-444a4df81149-etc-selinux\") pod \"aws-ebs-csi-driver-node-5mmp2\" (UID: \"08a3068f-f5f3-4086-8e21-444a4df81149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5mmp2" Sep 10 12:18:01.600253 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599664 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-host-kubelet\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.600253 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599689 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c502cf69-0e72-4829-bb53-cd23f5367666-sys\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.600253 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599716 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e6b25f25-3e5d-45f3-8c46-7884a14b981f-serviceca\") pod \"node-ca-68mrh\" (UID: \"e6b25f25-3e5d-45f3-8c46-7884a14b981f\") " pod="openshift-image-registry/node-ca-68mrh" Sep 10 12:18:01.600253 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599736 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c502cf69-0e72-4829-bb53-cd23f5367666-sys\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.600253 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599692 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/08a3068f-f5f3-4086-8e21-444a4df81149-sys-fs\") pod \"aws-ebs-csi-driver-node-5mmp2\" (UID: \"08a3068f-f5f3-4086-8e21-444a4df81149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5mmp2" Sep 10 12:18:01.600253 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599742 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-multus-socket-dir-parent\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.600253 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599621 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-system-cni-dir\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.600253 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599736 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/08a3068f-f5f3-4086-8e21-444a4df81149-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5mmp2\" (UID: \"08a3068f-f5f3-4086-8e21-444a4df81149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5mmp2" Sep 10 12:18:01.600253 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599753 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-node-log\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.600253 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599753 2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 10 12:18:01.601008 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599718 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-node-log\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.601008 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599842 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a76158d-6412-48e2-a091-51cc34552ce2-tmp-dir\") pod \"node-resolver-pkt25\" (UID: \"2a76158d-6412-48e2-a091-51cc34552ce2\") " pod="openshift-dns/node-resolver-pkt25" Sep 10 12:18:01.601008 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599881 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-host-run-netns\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.601008 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599906 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-host-run-ovn-kubernetes\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.601008 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599930 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-ovnkube-script-lib\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.601008 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599951 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-host-run-netns\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.601008 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599970 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5qpj\" (UniqueName: \"kubernetes.io/projected/08a3068f-f5f3-4086-8e21-444a4df81149-kube-api-access-w5qpj\") pod \"aws-ebs-csi-driver-node-5mmp2\" (UID: \"08a3068f-f5f3-4086-8e21-444a4df81149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5mmp2" Sep 10 12:18:01.601008 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599973 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-host-run-ovn-kubernetes\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.601008 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.599993 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-cnibin\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.601008 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600017 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-run-systemd\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.601008 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600041 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-etc-openvswitch\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.601008 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600063 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3cf0d46b-95a9-4187-9395-ebd8a11b5771-konnectivity-ca\") pod \"konnectivity-agent-zjw9d\" (UID: \"3cf0d46b-95a9-4187-9395-ebd8a11b5771\") " pod="kube-system/konnectivity-agent-zjw9d" Sep 10 12:18:01.601008 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600085 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/08a3068f-f5f3-4086-8e21-444a4df81149-socket-dir\") pod \"aws-ebs-csi-driver-node-5mmp2\" (UID: \"08a3068f-f5f3-4086-8e21-444a4df81149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5mmp2" Sep 10 12:18:01.601008 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600085 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-cnibin\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.601008 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600128 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a76158d-6412-48e2-a091-51cc34552ce2-tmp-dir\") pod \"node-resolver-pkt25\" (UID: \"2a76158d-6412-48e2-a091-51cc34552ce2\") " pod="openshift-dns/node-resolver-pkt25" Sep 10 12:18:01.601008 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600139 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-etc-openvswitch\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.601008 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600167 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vndkc\" (UniqueName: \"kubernetes.io/projected/6cda5c27-4386-49bd-8ad1-eb1e86535408-kube-api-access-vndkc\") pod \"iptables-alerter-ktr9w\" (UID: \"6cda5c27-4386-49bd-8ad1-eb1e86535408\") " pod="openshift-network-operator/iptables-alerter-ktr9w" Sep 10 12:18:01.601008 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600223 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-run-systemd\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.601793 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600250 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-run-openvswitch\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.601793 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600274 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e6b25f25-3e5d-45f3-8c46-7884a14b981f-host\") pod \"node-ca-68mrh\" (UID: \"e6b25f25-3e5d-45f3-8c46-7884a14b981f\") " pod="openshift-image-registry/node-ca-68mrh" Sep 10 12:18:01.601793 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600297 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-cni-binary-copy\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.601793 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600322 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-multus-conf-dir\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.601793 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600345 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-host-run-netns\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.601793 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600355 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/08a3068f-f5f3-4086-8e21-444a4df81149-socket-dir\") pod \"aws-ebs-csi-driver-node-5mmp2\" (UID: \"08a3068f-f5f3-4086-8e21-444a4df81149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5mmp2" Sep 10 12:18:01.601793 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600369 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.601793 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600398 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1d17b055-9484-48c5-bd77-90870f87d6ff-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ft7lg\" (UID: \"1d17b055-9484-48c5-bd77-90870f87d6ff\") " pod="openshift-multus/multus-additional-cni-plugins-ft7lg" Sep 10 12:18:01.601793 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600412 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-run-openvswitch\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.601793 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600415 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e6b25f25-3e5d-45f3-8c46-7884a14b981f-host\") pod \"node-ca-68mrh\" (UID: \"e6b25f25-3e5d-45f3-8c46-7884a14b981f\") " pod="openshift-image-registry/node-ca-68mrh" Sep 10 12:18:01.601793 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600430 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c502cf69-0e72-4829-bb53-cd23f5367666-etc-sysconfig\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.601793 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600454 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c502cf69-0e72-4829-bb53-cd23f5367666-etc-sysctl-d\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.601793 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600470 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-host-run-netns\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.601793 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600479 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c502cf69-0e72-4829-bb53-cd23f5367666-run\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.601793 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600502 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c502cf69-0e72-4829-bb53-cd23f5367666-lib-modules\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.601793 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600506 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-multus-conf-dir\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.601793 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600525 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c502cf69-0e72-4829-bb53-cd23f5367666-tmp\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.601793 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600532 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-ovnkube-script-lib\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.602724 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600550 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-host-cni-bin\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.602724 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600557 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c502cf69-0e72-4829-bb53-cd23f5367666-etc-sysconfig\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.602724 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600575 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/08a3068f-f5f3-4086-8e21-444a4df81149-device-dir\") pod \"aws-ebs-csi-driver-node-5mmp2\" (UID: \"08a3068f-f5f3-4086-8e21-444a4df81149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5mmp2" Sep 10 12:18:01.602724 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600600 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c502cf69-0e72-4829-bb53-cd23f5367666-etc-systemd\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.602724 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600609 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c502cf69-0e72-4829-bb53-cd23f5367666-run\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.602724 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600621 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fsrm\" (UniqueName: \"kubernetes.io/projected/2a76158d-6412-48e2-a091-51cc34552ce2-kube-api-access-2fsrm\") pod \"node-resolver-pkt25\" (UID: \"2a76158d-6412-48e2-a091-51cc34552ce2\") " pod="openshift-dns/node-resolver-pkt25" Sep 10 12:18:01.602724 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600628 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.602724 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600647 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-host-var-lib-cni-bin\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.602724 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600670 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-host-cni-netd\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.602724 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600697 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95vgz\" (UniqueName: \"kubernetes.io/projected/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-kube-api-access-95vgz\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.602724 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600729 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3cf0d46b-95a9-4187-9395-ebd8a11b5771-agent-certs\") pod \"konnectivity-agent-zjw9d\" (UID: \"3cf0d46b-95a9-4187-9395-ebd8a11b5771\") " pod="kube-system/konnectivity-agent-zjw9d" Sep 10 12:18:01.602724 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600739 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c502cf69-0e72-4829-bb53-cd23f5367666-etc-sysctl-d\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.602724 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600754 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c502cf69-0e72-4829-bb53-cd23f5367666-etc-sysctl-conf\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.602724 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600778 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c502cf69-0e72-4829-bb53-cd23f5367666-etc-tuned\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.602724 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600796 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c502cf69-0e72-4829-bb53-cd23f5367666-etc-systemd\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.602724 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600801 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtfw6\" (UniqueName: \"kubernetes.io/projected/c502cf69-0e72-4829-bb53-cd23f5367666-kube-api-access-vtfw6\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.602724 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600820 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3cf0d46b-95a9-4187-9395-ebd8a11b5771-konnectivity-ca\") pod \"konnectivity-agent-zjw9d\" (UID: \"3cf0d46b-95a9-4187-9395-ebd8a11b5771\") " pod="kube-system/konnectivity-agent-zjw9d" Sep 10 12:18:01.603625 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600826 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-etc-kubernetes\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.603625 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600851 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-env-overrides\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.603625 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600896 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6cda5c27-4386-49bd-8ad1-eb1e86535408-iptables-alerter-script\") pod \"iptables-alerter-ktr9w\" (UID: \"6cda5c27-4386-49bd-8ad1-eb1e86535408\") " pod="openshift-network-operator/iptables-alerter-ktr9w" Sep 10 12:18:01.603625 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600924 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1d17b055-9484-48c5-bd77-90870f87d6ff-system-cni-dir\") pod \"multus-additional-cni-plugins-ft7lg\" (UID: \"1d17b055-9484-48c5-bd77-90870f87d6ff\") " pod="openshift-multus/multus-additional-cni-plugins-ft7lg" Sep 10 12:18:01.603625 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600955 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1d17b055-9484-48c5-bd77-90870f87d6ff-cnibin\") pod \"multus-additional-cni-plugins-ft7lg\" (UID: \"1d17b055-9484-48c5-bd77-90870f87d6ff\") " pod="openshift-multus/multus-additional-cni-plugins-ft7lg" Sep 10 12:18:01.603625 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600974 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-cni-binary-copy\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.603625 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600980 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c502cf69-0e72-4829-bb53-cd23f5367666-etc-modprobe-d\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.603625 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.601023 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-host-var-lib-cni-bin\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.603625 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.601028 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c502cf69-0e72-4829-bb53-cd23f5367666-etc-kubernetes\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.603625 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.601060 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdh47\" (UniqueName: \"kubernetes.io/projected/7169f42e-6be8-4806-bf45-c77b6e2bdfd1-kube-api-access-rdh47\") pod \"network-metrics-daemon-7vx2g\" (UID: \"7169f42e-6be8-4806-bf45-c77b6e2bdfd1\") " pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:01.603625 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.601063 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c502cf69-0e72-4829-bb53-cd23f5367666-etc-modprobe-d\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.603625 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.601070 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c502cf69-0e72-4829-bb53-cd23f5367666-etc-kubernetes\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.603625 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.601080 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-host-cni-netd\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.603625 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.601085 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffmpn\" (UniqueName: \"kubernetes.io/projected/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-kube-api-access-ffmpn\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.603625 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.601410 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-etc-kubernetes\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.603625 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.601617 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1d17b055-9484-48c5-bd77-90870f87d6ff-system-cni-dir\") pod \"multus-additional-cni-plugins-ft7lg\" (UID: \"1d17b055-9484-48c5-bd77-90870f87d6ff\") " pod="openshift-multus/multus-additional-cni-plugins-ft7lg" Sep 10 12:18:01.603625 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.600900 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c502cf69-0e72-4829-bb53-cd23f5367666-lib-modules\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.604386 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.601646 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c502cf69-0e72-4829-bb53-cd23f5367666-etc-sysctl-conf\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.604386 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.601681 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1d17b055-9484-48c5-bd77-90870f87d6ff-cnibin\") pod \"multus-additional-cni-plugins-ft7lg\" (UID: \"1d17b055-9484-48c5-bd77-90870f87d6ff\") " pod="openshift-multus/multus-additional-cni-plugins-ft7lg" Sep 10 12:18:01.604386 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.601734 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/08a3068f-f5f3-4086-8e21-444a4df81149-device-dir\") pod \"aws-ebs-csi-driver-node-5mmp2\" (UID: \"08a3068f-f5f3-4086-8e21-444a4df81149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5mmp2" Sep 10 12:18:01.604386 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.601756 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-host-cni-bin\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.604386 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.601852 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-852dl\" (UniqueName: \"kubernetes.io/projected/1b89f971-1fee-46bf-ad4c-e68e6b83b9a0-kube-api-access-852dl\") pod \"network-check-target-d7ls5\" (UID: \"1b89f971-1fee-46bf-ad4c-e68e6b83b9a0\") " pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:18:01.604386 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.602091 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1d17b055-9484-48c5-bd77-90870f87d6ff-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ft7lg\" (UID: \"1d17b055-9484-48c5-bd77-90870f87d6ff\") " pod="openshift-multus/multus-additional-cni-plugins-ft7lg" Sep 10 12:18:01.604386 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.602118 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2a76158d-6412-48e2-a091-51cc34552ce2-hosts-file\") pod \"node-resolver-pkt25\" (UID: \"2a76158d-6412-48e2-a091-51cc34552ce2\") " pod="openshift-dns/node-resolver-pkt25" Sep 10 12:18:01.604386 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.602145 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-host-run-multus-certs\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.604386 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.602195 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-host-slash\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.604386 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.602226 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2a76158d-6412-48e2-a091-51cc34552ce2-hosts-file\") pod \"node-resolver-pkt25\" (UID: \"2a76158d-6412-48e2-a091-51cc34552ce2\") " pod="openshift-dns/node-resolver-pkt25" Sep 10 12:18:01.604386 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.602221 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-run-ovn\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.604386 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.602272 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-log-socket\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.604386 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.602289 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-host-run-multus-certs\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.604386 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.602304 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-ovnkube-config\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.604386 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.602335 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p79cx\" (UniqueName: \"kubernetes.io/projected/1d17b055-9484-48c5-bd77-90870f87d6ff-kube-api-access-p79cx\") pod \"multus-additional-cni-plugins-ft7lg\" (UID: \"1d17b055-9484-48c5-bd77-90870f87d6ff\") " pod="openshift-multus/multus-additional-cni-plugins-ft7lg" Sep 10 12:18:01.604386 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.602364 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6lf4\" (UniqueName: \"kubernetes.io/projected/e6b25f25-3e5d-45f3-8c46-7884a14b981f-kube-api-access-t6lf4\") pod \"node-ca-68mrh\" (UID: \"e6b25f25-3e5d-45f3-8c46-7884a14b981f\") " pod="openshift-image-registry/node-ca-68mrh" Sep 10 12:18:01.604386 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.602371 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-log-socket\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.604964 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.602322 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-host-slash\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.604964 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.602390 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-host-var-lib-cni-multus\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.604964 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.602336 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-run-ovn\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.604964 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.602558 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-env-overrides\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.604964 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.602584 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-host-var-lib-kubelet\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.604964 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.602633 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-multus-daemon-config\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.604964 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.602672 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6cda5c27-4386-49bd-8ad1-eb1e86535408-iptables-alerter-script\") pod \"iptables-alerter-ktr9w\" (UID: \"6cda5c27-4386-49bd-8ad1-eb1e86535408\") " pod="openshift-network-operator/iptables-alerter-ktr9w" Sep 10 12:18:01.604964 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.602762 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-host-var-lib-cni-multus\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.604964 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.602991 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1d17b055-9484-48c5-bd77-90870f87d6ff-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ft7lg\" (UID: \"1d17b055-9484-48c5-bd77-90870f87d6ff\") " pod="openshift-multus/multus-additional-cni-plugins-ft7lg" Sep 10 12:18:01.604964 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.603042 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-host-var-lib-kubelet\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.604964 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.603280 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-ovnkube-config\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.604964 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.603311 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-multus-daemon-config\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.604964 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.603510 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1d17b055-9484-48c5-bd77-90870f87d6ff-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ft7lg\" (UID: \"1d17b055-9484-48c5-bd77-90870f87d6ff\") " pod="openshift-multus/multus-additional-cni-plugins-ft7lg" Sep 10 12:18:01.604964 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.604040 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c502cf69-0e72-4829-bb53-cd23f5367666-tmp\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.604964 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.604147 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c502cf69-0e72-4829-bb53-cd23f5367666-etc-tuned\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.604964 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.604256 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-ovn-node-metrics-cert\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.604964 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.604313 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3cf0d46b-95a9-4187-9395-ebd8a11b5771-agent-certs\") pod \"konnectivity-agent-zjw9d\" (UID: \"3cf0d46b-95a9-4187-9395-ebd8a11b5771\") " pod="kube-system/konnectivity-agent-zjw9d" Sep 10 12:18:01.614628 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.614455 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vndkc\" (UniqueName: \"kubernetes.io/projected/6cda5c27-4386-49bd-8ad1-eb1e86535408-kube-api-access-vndkc\") pod \"iptables-alerter-ktr9w\" (UID: \"6cda5c27-4386-49bd-8ad1-eb1e86535408\") " pod="openshift-network-operator/iptables-alerter-ktr9w" Sep 10 12:18:01.615346 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:01.615317 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 12:18:01.615346 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:01.615348 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 12:18:01.615545 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:01.615365 2580 projected.go:196] Error preparing data for projected volume kube-api-access-852dl for pod openshift-network-diagnostics/network-check-target-d7ls5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:18:01.615545 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:01.615445 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b89f971-1fee-46bf-ad4c-e68e6b83b9a0-kube-api-access-852dl podName:1b89f971-1fee-46bf-ad4c-e68e6b83b9a0 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:02.115424178 +0000 UTC m=+2.122740539 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-852dl" (UniqueName: "kubernetes.io/projected/1b89f971-1fee-46bf-ad4c-e68e6b83b9a0-kube-api-access-852dl") pod "network-check-target-d7ls5" (UID: "1b89f971-1fee-46bf-ad4c-e68e6b83b9a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:18:01.616770 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.616742 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p79cx\" (UniqueName: \"kubernetes.io/projected/1d17b055-9484-48c5-bd77-90870f87d6ff-kube-api-access-p79cx\") pod \"multus-additional-cni-plugins-ft7lg\" (UID: \"1d17b055-9484-48c5-bd77-90870f87d6ff\") " pod="openshift-multus/multus-additional-cni-plugins-ft7lg" Sep 10 12:18:01.617110 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.617087 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtfw6\" (UniqueName: \"kubernetes.io/projected/c502cf69-0e72-4829-bb53-cd23f5367666-kube-api-access-vtfw6\") pod \"tuned-t8ths\" (UID: \"c502cf69-0e72-4829-bb53-cd23f5367666\") " pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.617203 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.617158 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffmpn\" (UniqueName: \"kubernetes.io/projected/0c94a7b9-9724-416f-8b9e-b98acd8c3c41-kube-api-access-ffmpn\") pod \"multus-5g7lk\" (UID: \"0c94a7b9-9724-416f-8b9e-b98acd8c3c41\") " pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.617618 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.617589 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fsrm\" (UniqueName: \"kubernetes.io/projected/2a76158d-6412-48e2-a091-51cc34552ce2-kube-api-access-2fsrm\") pod \"node-resolver-pkt25\" (UID: \"2a76158d-6412-48e2-a091-51cc34552ce2\") " pod="openshift-dns/node-resolver-pkt25" Sep 10 12:18:01.618070 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.618044 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6lf4\" (UniqueName: \"kubernetes.io/projected/e6b25f25-3e5d-45f3-8c46-7884a14b981f-kube-api-access-t6lf4\") pod \"node-ca-68mrh\" (UID: \"e6b25f25-3e5d-45f3-8c46-7884a14b981f\") " pod="openshift-image-registry/node-ca-68mrh" Sep 10 12:18:01.618283 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.618262 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdh47\" (UniqueName: \"kubernetes.io/projected/7169f42e-6be8-4806-bf45-c77b6e2bdfd1-kube-api-access-rdh47\") pod \"network-metrics-daemon-7vx2g\" (UID: \"7169f42e-6be8-4806-bf45-c77b6e2bdfd1\") " pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:01.618492 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.618473 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95vgz\" (UniqueName: \"kubernetes.io/projected/e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58-kube-api-access-95vgz\") pod \"ovnkube-node-tfcmz\" (UID: \"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.618731 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.618712 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5qpj\" (UniqueName: \"kubernetes.io/projected/08a3068f-f5f3-4086-8e21-444a4df81149-kube-api-access-w5qpj\") pod \"aws-ebs-csi-driver-node-5mmp2\" (UID: \"08a3068f-f5f3-4086-8e21-444a4df81149\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5mmp2" Sep 10 12:18:01.664094 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.664019 2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 12:18:01.693389 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.693357 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 12:18:01.710949 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.710917 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zjw9d" Sep 10 12:18:01.716917 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:01.716878 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbeff53fae53b2340a3f2a4afbf5b2aa.slice/crio-05e298dfc8077ed88c3fb994f2b2065972cfd04ba065c718fe4f961202d1bc9f WatchSource:0}: Error finding container 05e298dfc8077ed88c3fb994f2b2065972cfd04ba065c718fe4f961202d1bc9f: Status 404 returned error can't find the container with id 05e298dfc8077ed88c3fb994f2b2065972cfd04ba065c718fe4f961202d1bc9f Sep 10 12:18:01.721907 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.721885 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 12:18:01.722007 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:01.721903 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cf0d46b_95a9_4187_9395_ebd8a11b5771.slice/crio-242b193324018805aa541176b526a77925c528f4ce8595605348c959cbb58c07 WatchSource:0}: Error finding container 242b193324018805aa541176b526a77925c528f4ce8595605348c959cbb58c07: Status 404 returned error can't find the container with id 242b193324018805aa541176b526a77925c528f4ce8595605348c959cbb58c07 Sep 10 12:18:01.732089 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.732067 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5mmp2" Sep 10 12:18:01.740769 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:01.740738 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08a3068f_f5f3_4086_8e21_444a4df81149.slice/crio-a214f9aeb33d5edc4285f32bf4da181ad67039c7daec0cb3be89762ec0dd8c73 WatchSource:0}: Error finding container a214f9aeb33d5edc4285f32bf4da181ad67039c7daec0cb3be89762ec0dd8c73: Status 404 returned error can't find the container with id a214f9aeb33d5edc4285f32bf4da181ad67039c7daec0cb3be89762ec0dd8c73 Sep 10 12:18:01.752467 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.752443 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-t8ths" Sep 10 12:18:01.760339 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:01.760293 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4983c0fc993a6c067676ddbfae0d2d6.slice/crio-240225c83b5e6415eb01bdf466e7c5cf01bf58692833e5c4c87b84b6e4f85258 WatchSource:0}: Error finding container 240225c83b5e6415eb01bdf466e7c5cf01bf58692833e5c4c87b84b6e4f85258: Status 404 returned error can't find the container with id 240225c83b5e6415eb01bdf466e7c5cf01bf58692833e5c4c87b84b6e4f85258 Sep 10 12:18:01.761241 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:01.761217 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc502cf69_0e72_4829_bb53_cd23f5367666.slice/crio-8c1c9eef865e4a86b1745e14e4de4df09fcf820c97e9145f13dcb1dc9168d97c WatchSource:0}: Error finding container 8c1c9eef865e4a86b1745e14e4de4df09fcf820c97e9145f13dcb1dc9168d97c: Status 404 returned error can't find the container with id 8c1c9eef865e4a86b1745e14e4de4df09fcf820c97e9145f13dcb1dc9168d97c Sep 10 12:18:01.769063 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.769038 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pkt25" Sep 10 12:18:01.780808 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:01.780774 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a76158d_6412_48e2_a091_51cc34552ce2.slice/crio-a9d455577f1b790afc10df1155b64b5b11e7e342393504fcd5851d77d4a31e28 WatchSource:0}: Error finding container a9d455577f1b790afc10df1155b64b5b11e7e342393504fcd5851d77d4a31e28: Status 404 returned error can't find the container with id a9d455577f1b790afc10df1155b64b5b11e7e342393504fcd5851d77d4a31e28 Sep 10 12:18:01.782754 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.782731 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-68mrh" Sep 10 12:18:01.788406 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.788381 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ktr9w" Sep 10 12:18:01.791534 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:01.791504 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6b25f25_3e5d_45f3_8c46_7884a14b981f.slice/crio-120b5712c5cc36175ceae575b139132a74dc5fc72efcc9af32ec35db48f59003 WatchSource:0}: Error finding container 120b5712c5cc36175ceae575b139132a74dc5fc72efcc9af32ec35db48f59003: Status 404 returned error can't find the container with id 120b5712c5cc36175ceae575b139132a74dc5fc72efcc9af32ec35db48f59003 Sep 10 12:18:01.797228 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:01.797195 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cda5c27_4386_49bd_8ad1_eb1e86535408.slice/crio-5b744a67e6350e321f8963d0f5f2c120ae4eea20b9fc3039066e2c98f0f4b793 WatchSource:0}: Error finding container 5b744a67e6350e321f8963d0f5f2c120ae4eea20b9fc3039066e2c98f0f4b793: Status 404 returned error can't find the container with id 5b744a67e6350e321f8963d0f5f2c120ae4eea20b9fc3039066e2c98f0f4b793 Sep 10 12:18:01.804462 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.804436 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5g7lk" Sep 10 12:18:01.810469 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.810447 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ft7lg" Sep 10 12:18:01.814402 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:01.814375 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c94a7b9_9724_416f_8b9e_b98acd8c3c41.slice/crio-d8cdf05a685e9d4e144a84280951302e6d210afcf53d4e156f8fecb877eecf2e WatchSource:0}: Error finding container d8cdf05a685e9d4e144a84280951302e6d210afcf53d4e156f8fecb877eecf2e: Status 404 returned error can't find the container with id d8cdf05a685e9d4e144a84280951302e6d210afcf53d4e156f8fecb877eecf2e Sep 10 12:18:01.819258 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:01.819232 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d17b055_9484_48c5_bd77_90870f87d6ff.slice/crio-e3006426263e50981d043c49468b14d0aba88280e2331e0f563cf6b9981bc7ab WatchSource:0}: Error finding container e3006426263e50981d043c49468b14d0aba88280e2331e0f563cf6b9981bc7ab: Status 404 returned error can't find the container with id e3006426263e50981d043c49468b14d0aba88280e2331e0f563cf6b9981bc7ab Sep 10 12:18:01.828501 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:01.828471 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:01.838694 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:18:01.838435 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode828e8f0_b7fe_4d1c_850e_ffcc8ee4bd58.slice/crio-036a4ea615ac31a8e017de2319de5e9e6b43dd725ee9cd9d6bfed4272e0f377e WatchSource:0}: Error finding container 036a4ea615ac31a8e017de2319de5e9e6b43dd725ee9cd9d6bfed4272e0f377e: Status 404 returned error can't find the container with id 036a4ea615ac31a8e017de2319de5e9e6b43dd725ee9cd9d6bfed4272e0f377e Sep 10 12:18:01.840464 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:01.840442 2580 kuberuntime_manager.go:1459] "Unhandled Error" err="container ovn-controller start failed in pod ovnkube-node-tfcmz_openshift-ovn-kubernetes(e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Sep 10 12:18:01.842020 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:01.841998 2580 kuberuntime_manager.go:1459] "Unhandled Error" err="container kube-rbac-proxy-node start failed in pod ovnkube-node-tfcmz_openshift-ovn-kubernetes(e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Sep 10 12:18:01.845301 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:01.845249 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"ovn-controller\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"ovn-acl-logging\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ff86545c65970d36d4ced99fee46fd9a5a7590e379152c8c728eb1f29970dce0\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-node\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-ovn-metrics\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cec847b1f08cae0315bd44fe477d725a0180b78ab0ad844de3b218bf2dde0bb2\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"northd\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ff86545c65970d36d4ced99fee46fd9a5a7590e379152c8c728eb1f29970dce0\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"nbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ff86545c65970d36d4ced99fee46fd9a5a7590e379152c8c728eb1f29970dce0\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"sbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ff86545c65970d36d4ced99fee46fd9a5a7590e379152c8c728eb1f29970dce0\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"ovnkube-controller\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ff86545c65970d36d4ced99fee46fd9a5a7590e379152c8c728eb1f29970dce0\\\": ErrImagePull: pull QPS exceeded\"]" pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" podUID="e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58" Sep 10 12:18:02.006613 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:02.006499 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/67095b3f-a8e2-494b-9f83-b4eb0b5634a2-original-pull-secret\") pod \"global-pull-secret-syncer-ljkj8\" (UID: \"67095b3f-a8e2-494b-9f83-b4eb0b5634a2\") " pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:02.006809 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:02.006639 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 12:18:02.006809 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:02.006713 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67095b3f-a8e2-494b-9f83-b4eb0b5634a2-original-pull-secret podName:67095b3f-a8e2-494b-9f83-b4eb0b5634a2 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:03.006694724 +0000 UTC m=+3.014011084 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/67095b3f-a8e2-494b-9f83-b4eb0b5634a2-original-pull-secret") pod "global-pull-secret-syncer-ljkj8" (UID: "67095b3f-a8e2-494b-9f83-b4eb0b5634a2") : object "kube-system"/"original-pull-secret" not registered Sep 10 12:18:02.106950 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:02.106910 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7169f42e-6be8-4806-bf45-c77b6e2bdfd1-metrics-certs\") pod \"network-metrics-daemon-7vx2g\" (UID: \"7169f42e-6be8-4806-bf45-c77b6e2bdfd1\") " pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:02.107149 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:02.107047 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:18:02.107149 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:02.107115 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7169f42e-6be8-4806-bf45-c77b6e2bdfd1-metrics-certs podName:7169f42e-6be8-4806-bf45-c77b6e2bdfd1 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:03.107095163 +0000 UTC m=+3.114411524 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7169f42e-6be8-4806-bf45-c77b6e2bdfd1-metrics-certs") pod "network-metrics-daemon-7vx2g" (UID: "7169f42e-6be8-4806-bf45-c77b6e2bdfd1") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:18:02.207809 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:02.207766 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-852dl\" (UniqueName: \"kubernetes.io/projected/1b89f971-1fee-46bf-ad4c-e68e6b83b9a0-kube-api-access-852dl\") pod \"network-check-target-d7ls5\" (UID: \"1b89f971-1fee-46bf-ad4c-e68e6b83b9a0\") " pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:18:02.208058 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:02.207993 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 12:18:02.208058 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:02.208019 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 12:18:02.208058 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:02.208033 2580 projected.go:196] Error preparing data for projected volume kube-api-access-852dl for pod openshift-network-diagnostics/network-check-target-d7ls5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:18:02.208217 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:02.208118 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b89f971-1fee-46bf-ad4c-e68e6b83b9a0-kube-api-access-852dl podName:1b89f971-1fee-46bf-ad4c-e68e6b83b9a0 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:03.208087618 +0000 UTC m=+3.215403993 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-852dl" (UniqueName: "kubernetes.io/projected/1b89f971-1fee-46bf-ad4c-e68e6b83b9a0-kube-api-access-852dl") pod "network-check-target-d7ls5" (UID: "1b89f971-1fee-46bf-ad4c-e68e6b83b9a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:18:02.447352 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:02.447251 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 12:13:01 +0000 UTC" deadline="2028-03-02 19:57:20.310435057 +0000 UTC" Sep 10 12:18:02.447352 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:02.447292 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12943h39m17.86314733s" Sep 10 12:18:02.459144 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:02.459107 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 12:18:02.592628 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:02.592329 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-221.ec2.internal" event={"ID":"b4983c0fc993a6c067676ddbfae0d2d6","Type":"ContainerStarted","Data":"240225c83b5e6415eb01bdf466e7c5cf01bf58692833e5c4c87b84b6e4f85258"} Sep 10 12:18:02.595175 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:02.595124 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5mmp2" event={"ID":"08a3068f-f5f3-4086-8e21-444a4df81149","Type":"ContainerStarted","Data":"a214f9aeb33d5edc4285f32bf4da181ad67039c7daec0cb3be89762ec0dd8c73"} Sep 10 12:18:02.601019 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:02.600921 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ktr9w" event={"ID":"6cda5c27-4386-49bd-8ad1-eb1e86535408","Type":"ContainerStarted","Data":"5b744a67e6350e321f8963d0f5f2c120ae4eea20b9fc3039066e2c98f0f4b793"} Sep 10 12:18:02.612552 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:02.612502 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zjw9d" event={"ID":"3cf0d46b-95a9-4187-9395-ebd8a11b5771","Type":"ContainerStarted","Data":"242b193324018805aa541176b526a77925c528f4ce8595605348c959cbb58c07"} Sep 10 12:18:02.618447 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:02.618395 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-221.ec2.internal" event={"ID":"dbeff53fae53b2340a3f2a4afbf5b2aa","Type":"ContainerStarted","Data":"05e298dfc8077ed88c3fb994f2b2065972cfd04ba065c718fe4f961202d1bc9f"} Sep 10 12:18:02.631355 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:02.631304 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" event={"ID":"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58","Type":"ContainerStarted","Data":"036a4ea615ac31a8e017de2319de5e9e6b43dd725ee9cd9d6bfed4272e0f377e"} Sep 10 12:18:02.638206 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:02.638117 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ft7lg" event={"ID":"1d17b055-9484-48c5-bd77-90870f87d6ff","Type":"ContainerStarted","Data":"e3006426263e50981d043c49468b14d0aba88280e2331e0f563cf6b9981bc7ab"} Sep 10 12:18:02.644095 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:02.644060 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5g7lk" event={"ID":"0c94a7b9-9724-416f-8b9e-b98acd8c3c41","Type":"ContainerStarted","Data":"d8cdf05a685e9d4e144a84280951302e6d210afcf53d4e156f8fecb877eecf2e"} Sep 10 12:18:02.648791 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:02.648747 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-68mrh" event={"ID":"e6b25f25-3e5d-45f3-8c46-7884a14b981f","Type":"ContainerStarted","Data":"120b5712c5cc36175ceae575b139132a74dc5fc72efcc9af32ec35db48f59003"} Sep 10 12:18:02.661108 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:02.661069 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pkt25" event={"ID":"2a76158d-6412-48e2-a091-51cc34552ce2","Type":"ContainerStarted","Data":"a9d455577f1b790afc10df1155b64b5b11e7e342393504fcd5851d77d4a31e28"} Sep 10 12:18:02.668215 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:02.668128 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"ovn-controller\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ff86545c65970d36d4ced99fee46fd9a5a7590e379152c8c728eb1f29970dce0\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"ovn-acl-logging\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ff86545c65970d36d4ced99fee46fd9a5a7590e379152c8c728eb1f29970dce0\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-node\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cec847b1f08cae0315bd44fe477d725a0180b78ab0ad844de3b218bf2dde0bb2\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-ovn-metrics\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cec847b1f08cae0315bd44fe477d725a0180b78ab0ad844de3b218bf2dde0bb2\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"northd\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ff86545c65970d36d4ced99fee46fd9a5a7590e379152c8c728eb1f29970dce0\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"nbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ff86545c65970d36d4ced99fee46fd9a5a7590e379152c8c728eb1f29970dce0\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"sbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ff86545c65970d36d4ced99fee46fd9a5a7590e379152c8c728eb1f29970dce0\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"ovnkube-controller\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ff86545c65970d36d4ced99fee46fd9a5a7590e379152c8c728eb1f29970dce0\\\": ErrImagePull: pull QPS exceeded\"]" pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" podUID="e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58" Sep 10 12:18:02.680798 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:02.680752 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-t8ths" event={"ID":"c502cf69-0e72-4829-bb53-cd23f5367666","Type":"ContainerStarted","Data":"8c1c9eef865e4a86b1745e14e4de4df09fcf820c97e9145f13dcb1dc9168d97c"} Sep 10 12:18:03.016704 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:03.016655 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/67095b3f-a8e2-494b-9f83-b4eb0b5634a2-original-pull-secret\") pod \"global-pull-secret-syncer-ljkj8\" (UID: \"67095b3f-a8e2-494b-9f83-b4eb0b5634a2\") " pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:03.016965 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:03.016834 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 12:18:03.016965 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:03.016927 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67095b3f-a8e2-494b-9f83-b4eb0b5634a2-original-pull-secret podName:67095b3f-a8e2-494b-9f83-b4eb0b5634a2 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:05.016884085 +0000 UTC m=+5.024200463 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/67095b3f-a8e2-494b-9f83-b4eb0b5634a2-original-pull-secret") pod "global-pull-secret-syncer-ljkj8" (UID: "67095b3f-a8e2-494b-9f83-b4eb0b5634a2") : object "kube-system"/"original-pull-secret" not registered Sep 10 12:18:03.117728 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:03.117663 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7169f42e-6be8-4806-bf45-c77b6e2bdfd1-metrics-certs\") pod \"network-metrics-daemon-7vx2g\" (UID: \"7169f42e-6be8-4806-bf45-c77b6e2bdfd1\") " pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:03.117972 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:03.117887 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:18:03.118073 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:03.118025 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7169f42e-6be8-4806-bf45-c77b6e2bdfd1-metrics-certs podName:7169f42e-6be8-4806-bf45-c77b6e2bdfd1 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:05.118003687 +0000 UTC m=+5.125320070 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7169f42e-6be8-4806-bf45-c77b6e2bdfd1-metrics-certs") pod "network-metrics-daemon-7vx2g" (UID: "7169f42e-6be8-4806-bf45-c77b6e2bdfd1") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:18:03.218434 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:03.218388 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-852dl\" (UniqueName: \"kubernetes.io/projected/1b89f971-1fee-46bf-ad4c-e68e6b83b9a0-kube-api-access-852dl\") pod \"network-check-target-d7ls5\" (UID: \"1b89f971-1fee-46bf-ad4c-e68e6b83b9a0\") " pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:18:03.218682 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:03.218648 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 12:18:03.218682 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:03.218680 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 12:18:03.218815 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:03.218696 2580 projected.go:196] Error preparing data for projected volume kube-api-access-852dl for pod openshift-network-diagnostics/network-check-target-d7ls5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:18:03.218815 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:03.218764 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b89f971-1fee-46bf-ad4c-e68e6b83b9a0-kube-api-access-852dl podName:1b89f971-1fee-46bf-ad4c-e68e6b83b9a0 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:05.218743461 +0000 UTC m=+5.226059830 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-852dl" (UniqueName: "kubernetes.io/projected/1b89f971-1fee-46bf-ad4c-e68e6b83b9a0-kube-api-access-852dl") pod "network-check-target-d7ls5" (UID: "1b89f971-1fee-46bf-ad4c-e68e6b83b9a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:18:03.447986 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:03.447818 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 12:13:01 +0000 UTC" deadline="2028-02-25 18:27:54.611465216 +0000 UTC" Sep 10 12:18:03.447986 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:03.447909 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12798h9m51.163561501s" Sep 10 12:18:03.551524 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:03.551474 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:03.551796 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:03.551673 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkj8" podUID="67095b3f-a8e2-494b-9f83-b4eb0b5634a2" Sep 10 12:18:03.552217 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:03.552184 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:03.552333 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:03.552310 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vx2g" podUID="7169f42e-6be8-4806-bf45-c77b6e2bdfd1" Sep 10 12:18:03.552397 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:03.552379 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:18:03.552478 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:03.552457 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d7ls5" podUID="1b89f971-1fee-46bf-ad4c-e68e6b83b9a0" Sep 10 12:18:05.037523 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:05.036211 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/67095b3f-a8e2-494b-9f83-b4eb0b5634a2-original-pull-secret\") pod \"global-pull-secret-syncer-ljkj8\" (UID: \"67095b3f-a8e2-494b-9f83-b4eb0b5634a2\") " pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:05.037523 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:05.036419 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 12:18:05.037523 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:05.036485 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67095b3f-a8e2-494b-9f83-b4eb0b5634a2-original-pull-secret podName:67095b3f-a8e2-494b-9f83-b4eb0b5634a2 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:09.036466845 +0000 UTC m=+9.043783230 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/67095b3f-a8e2-494b-9f83-b4eb0b5634a2-original-pull-secret") pod "global-pull-secret-syncer-ljkj8" (UID: "67095b3f-a8e2-494b-9f83-b4eb0b5634a2") : object "kube-system"/"original-pull-secret" not registered Sep 10 12:18:05.137267 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:05.137220 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7169f42e-6be8-4806-bf45-c77b6e2bdfd1-metrics-certs\") pod \"network-metrics-daemon-7vx2g\" (UID: \"7169f42e-6be8-4806-bf45-c77b6e2bdfd1\") " pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:05.137473 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:05.137404 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:18:05.137557 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:05.137489 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7169f42e-6be8-4806-bf45-c77b6e2bdfd1-metrics-certs podName:7169f42e-6be8-4806-bf45-c77b6e2bdfd1 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:09.137467257 +0000 UTC m=+9.144783621 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7169f42e-6be8-4806-bf45-c77b6e2bdfd1-metrics-certs") pod "network-metrics-daemon-7vx2g" (UID: "7169f42e-6be8-4806-bf45-c77b6e2bdfd1") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:18:05.238315 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:05.238273 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-852dl\" (UniqueName: \"kubernetes.io/projected/1b89f971-1fee-46bf-ad4c-e68e6b83b9a0-kube-api-access-852dl\") pod \"network-check-target-d7ls5\" (UID: \"1b89f971-1fee-46bf-ad4c-e68e6b83b9a0\") " pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:18:05.238552 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:05.238503 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 12:18:05.238552 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:05.238527 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 12:18:05.238552 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:05.238542 2580 projected.go:196] Error preparing data for projected volume kube-api-access-852dl for pod openshift-network-diagnostics/network-check-target-d7ls5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:18:05.238786 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:05.238616 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b89f971-1fee-46bf-ad4c-e68e6b83b9a0-kube-api-access-852dl podName:1b89f971-1fee-46bf-ad4c-e68e6b83b9a0 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:09.238597102 +0000 UTC m=+9.245913463 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-852dl" (UniqueName: "kubernetes.io/projected/1b89f971-1fee-46bf-ad4c-e68e6b83b9a0-kube-api-access-852dl") pod "network-check-target-d7ls5" (UID: "1b89f971-1fee-46bf-ad4c-e68e6b83b9a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:18:05.552121 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:05.552029 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:05.552121 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:05.552064 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:05.552121 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:05.552065 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:18:05.552445 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:05.552183 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkj8" podUID="67095b3f-a8e2-494b-9f83-b4eb0b5634a2" Sep 10 12:18:05.552445 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:05.552311 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vx2g" podUID="7169f42e-6be8-4806-bf45-c77b6e2bdfd1" Sep 10 12:18:05.552445 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:05.552406 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d7ls5" podUID="1b89f971-1fee-46bf-ad4c-e68e6b83b9a0" Sep 10 12:18:07.552348 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:07.552295 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:07.552348 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:07.552334 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:18:07.552965 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:07.552334 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:07.552965 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:07.552458 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkj8" podUID="67095b3f-a8e2-494b-9f83-b4eb0b5634a2" Sep 10 12:18:07.552965 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:07.552583 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d7ls5" podUID="1b89f971-1fee-46bf-ad4c-e68e6b83b9a0" Sep 10 12:18:07.552965 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:07.552680 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vx2g" podUID="7169f42e-6be8-4806-bf45-c77b6e2bdfd1" Sep 10 12:18:09.073278 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:09.073237 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/67095b3f-a8e2-494b-9f83-b4eb0b5634a2-original-pull-secret\") pod \"global-pull-secret-syncer-ljkj8\" (UID: \"67095b3f-a8e2-494b-9f83-b4eb0b5634a2\") " pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:09.073779 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:09.073416 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 12:18:09.073779 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:09.073507 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67095b3f-a8e2-494b-9f83-b4eb0b5634a2-original-pull-secret podName:67095b3f-a8e2-494b-9f83-b4eb0b5634a2 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:17.073484391 +0000 UTC m=+17.080800765 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/67095b3f-a8e2-494b-9f83-b4eb0b5634a2-original-pull-secret") pod "global-pull-secret-syncer-ljkj8" (UID: "67095b3f-a8e2-494b-9f83-b4eb0b5634a2") : object "kube-system"/"original-pull-secret" not registered Sep 10 12:18:09.173692 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:09.173653 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7169f42e-6be8-4806-bf45-c77b6e2bdfd1-metrics-certs\") pod \"network-metrics-daemon-7vx2g\" (UID: \"7169f42e-6be8-4806-bf45-c77b6e2bdfd1\") " pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:09.173904 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:09.173834 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:18:09.173974 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:09.173931 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7169f42e-6be8-4806-bf45-c77b6e2bdfd1-metrics-certs podName:7169f42e-6be8-4806-bf45-c77b6e2bdfd1 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:17.173907222 +0000 UTC m=+17.181223607 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7169f42e-6be8-4806-bf45-c77b6e2bdfd1-metrics-certs") pod "network-metrics-daemon-7vx2g" (UID: "7169f42e-6be8-4806-bf45-c77b6e2bdfd1") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:18:09.274162 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:09.274119 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-852dl\" (UniqueName: \"kubernetes.io/projected/1b89f971-1fee-46bf-ad4c-e68e6b83b9a0-kube-api-access-852dl\") pod \"network-check-target-d7ls5\" (UID: \"1b89f971-1fee-46bf-ad4c-e68e6b83b9a0\") " pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:18:09.274511 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:09.274484 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 12:18:09.274511 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:09.274516 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 12:18:09.274680 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:09.274530 2580 projected.go:196] Error preparing data for projected volume kube-api-access-852dl for pod openshift-network-diagnostics/network-check-target-d7ls5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:18:09.274680 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:09.274595 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b89f971-1fee-46bf-ad4c-e68e6b83b9a0-kube-api-access-852dl podName:1b89f971-1fee-46bf-ad4c-e68e6b83b9a0 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:17.274575212 +0000 UTC m=+17.281891587 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-852dl" (UniqueName: "kubernetes.io/projected/1b89f971-1fee-46bf-ad4c-e68e6b83b9a0-kube-api-access-852dl") pod "network-check-target-d7ls5" (UID: "1b89f971-1fee-46bf-ad4c-e68e6b83b9a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:18:09.551651 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:09.551550 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:18:09.551920 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:09.551550 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:09.551920 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:09.551703 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d7ls5" podUID="1b89f971-1fee-46bf-ad4c-e68e6b83b9a0" Sep 10 12:18:09.551920 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:09.551743 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkj8" podUID="67095b3f-a8e2-494b-9f83-b4eb0b5634a2" Sep 10 12:18:09.551920 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:09.551550 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:09.551920 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:09.551830 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vx2g" podUID="7169f42e-6be8-4806-bf45-c77b6e2bdfd1" Sep 10 12:18:11.552785 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:11.552442 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:11.552785 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:11.552486 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:11.553889 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:11.552803 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkj8" podUID="67095b3f-a8e2-494b-9f83-b4eb0b5634a2" Sep 10 12:18:11.553889 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:11.552486 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:18:11.553889 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:11.552894 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vx2g" podUID="7169f42e-6be8-4806-bf45-c77b6e2bdfd1" Sep 10 12:18:11.553889 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:11.552941 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d7ls5" podUID="1b89f971-1fee-46bf-ad4c-e68e6b83b9a0" Sep 10 12:18:11.705433 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:11.705204 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zjw9d" event={"ID":"3cf0d46b-95a9-4187-9395-ebd8a11b5771","Type":"ContainerStarted","Data":"3b7edc5c662e9309afadb0a62d053580019a471c26f302a7d5caa0fbf10377e4"} Sep 10 12:18:11.707149 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:11.707113 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-221.ec2.internal" event={"ID":"dbeff53fae53b2340a3f2a4afbf5b2aa","Type":"ContainerStarted","Data":"30c6b4df9ae6acb6327e7f5aa8ba09f8f14bd87247c77122d61b6c68fd986457"} Sep 10 12:18:11.709028 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:11.708996 2580 generic.go:356] "Generic (PLEG): container finished" podID="1d17b055-9484-48c5-bd77-90870f87d6ff" containerID="44e9cee39ffefcbbfcd0796ce969b4a60f96f6b8443249951c4f33afb43ae2b5" exitCode=0 Sep 10 12:18:11.709199 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:11.709078 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ft7lg" event={"ID":"1d17b055-9484-48c5-bd77-90870f87d6ff","Type":"ContainerDied","Data":"44e9cee39ffefcbbfcd0796ce969b4a60f96f6b8443249951c4f33afb43ae2b5"} Sep 10 12:18:11.711011 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:11.710982 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-68mrh" event={"ID":"e6b25f25-3e5d-45f3-8c46-7884a14b981f","Type":"ContainerStarted","Data":"e362e00a60779c56d9b58b41eedf20fefdd2e3190a825f8043811c9839bdf5a9"} Sep 10 12:18:11.712777 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:11.712750 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pkt25" event={"ID":"2a76158d-6412-48e2-a091-51cc34552ce2","Type":"ContainerStarted","Data":"7341755367a1f4b9a4f019b4bf0c3e61881aec0f0b799bc97f2718a41d5e490a"} Sep 10 12:18:11.714585 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:11.714561 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-t8ths" event={"ID":"c502cf69-0e72-4829-bb53-cd23f5367666","Type":"ContainerStarted","Data":"8f9edf1f86563d7909f7e23e0189f8f931d9883a9edc3eb86279a45a39b2865b"} Sep 10 12:18:11.716387 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:11.716361 2580 generic.go:356] "Generic (PLEG): container finished" podID="b4983c0fc993a6c067676ddbfae0d2d6" containerID="055a2469a1af90e891d02fbda639e8360db1b741d24f70b10ac3931d5bab5ca1" exitCode=0 Sep 10 12:18:11.716500 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:11.716439 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-221.ec2.internal" event={"ID":"b4983c0fc993a6c067676ddbfae0d2d6","Type":"ContainerDied","Data":"055a2469a1af90e891d02fbda639e8360db1b741d24f70b10ac3931d5bab5ca1"} Sep 10 12:18:11.718561 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:11.718535 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5mmp2" event={"ID":"08a3068f-f5f3-4086-8e21-444a4df81149","Type":"ContainerStarted","Data":"c5819e9a49f271db3220be9b3fe1babd39ad0e3bf89e15be370cec7928134b3c"} Sep 10 12:18:11.723254 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:11.723187 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-zjw9d" podStartSLOduration=2.452060818 podStartE2EDuration="11.72316769s" totalImagesPullingTime="9.271106872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:18:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:18:11.722706429 +0000 UTC m=+11.730022813" watchObservedRunningTime="2026-09-10 12:18:11.72316769 +0000 UTC m=+11.730484074" Sep 10 12:18:11.741765 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:11.741701 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-t8ths" podStartSLOduration=2.482452167 podStartE2EDuration="11.74168005s" totalImagesPullingTime="9.259227883s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:18:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:18:11.741558254 +0000 UTC m=+11.748874638" watchObservedRunningTime="2026-09-10 12:18:11.74168005 +0000 UTC m=+11.748996435" Sep 10 12:18:11.758837 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:11.758777 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-pkt25" podStartSLOduration=2.5456383110000003 podStartE2EDuration="11.758759068s" totalImagesPullingTime="9.213120757s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:18:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:18:11.7581799 +0000 UTC m=+11.765496285" watchObservedRunningTime="2026-09-10 12:18:11.758759068 +0000 UTC m=+11.766075451" Sep 10 12:18:11.776749 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:11.776691 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-68mrh" podStartSLOduration=2.584763782 podStartE2EDuration="11.776669899s" totalImagesPullingTime="9.191906117s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:18:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:18:11.776007202 +0000 UTC m=+11.783323583" watchObservedRunningTime="2026-09-10 12:18:11.776669899 +0000 UTC m=+11.783986282" Sep 10 12:18:11.844466 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:11.844350 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-221.ec2.internal" podStartSLOduration=10.844292729 podStartE2EDuration="10.844292729s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:18:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:18:11.842980484 +0000 UTC m=+11.850296869" watchObservedRunningTime="2026-09-10 12:18:11.844292729 +0000 UTC m=+11.851609115" Sep 10 12:18:12.338767 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:12.338739 2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 10 12:18:12.452455 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:12.452327 2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T12:18:12.33876246Z","UUID":"b5b2a2c7-0c80-4751-975a-815f6b506517","Handler":null,"Name":"","Endpoint":""} Sep 10 12:18:12.453907 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:12.453884 2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 10 12:18:12.454060 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:12.453922 2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 10 12:18:12.723347 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:12.723309 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-221.ec2.internal" event={"ID":"b4983c0fc993a6c067676ddbfae0d2d6","Type":"ContainerStarted","Data":"91db3dda6089d959ad59453823b6203320f7efdbb240a331ad999e9adff264fc"} Sep 10 12:18:12.725590 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:12.725554 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5mmp2" event={"ID":"08a3068f-f5f3-4086-8e21-444a4df81149","Type":"ContainerStarted","Data":"ede16f08f03b33e9459c5e3cc708d88553f08b93f81ef19c8839772f676e4bb7"} Sep 10 12:18:12.745500 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:12.745422 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-221.ec2.internal" podStartSLOduration=11.745397469 podStartE2EDuration="11.745397469s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:18:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:18:12.745020621 +0000 UTC m=+12.752337004" watchObservedRunningTime="2026-09-10 12:18:12.745397469 +0000 UTC m=+12.752713854" Sep 10 12:18:13.551642 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:13.551599 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:18:13.551873 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:13.551653 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:13.551873 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:13.551599 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:13.551873 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:13.551751 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d7ls5" podUID="1b89f971-1fee-46bf-ad4c-e68e6b83b9a0" Sep 10 12:18:13.551873 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:13.551797 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkj8" podUID="67095b3f-a8e2-494b-9f83-b4eb0b5634a2" Sep 10 12:18:13.552088 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:13.551905 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vx2g" podUID="7169f42e-6be8-4806-bf45-c77b6e2bdfd1" Sep 10 12:18:13.729998 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:13.729947 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ktr9w" event={"ID":"6cda5c27-4386-49bd-8ad1-eb1e86535408","Type":"ContainerStarted","Data":"6b1208c8f7beaf5934d4b8cce562ba45bd364485d951b0e1a7f0ff9f73eb8eda"} Sep 10 12:18:13.747305 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:13.747240 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-ktr9w" podStartSLOduration=4.55141291 podStartE2EDuration="13.74721761s" totalImagesPullingTime="9.1958047s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:18:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:18:13.747183386 +0000 UTC m=+13.754499769" watchObservedRunningTime="2026-09-10 12:18:13.74721761 +0000 UTC m=+13.754533974" Sep 10 12:18:15.343088 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:15.343036 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-zjw9d" Sep 10 12:18:15.343790 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:15.343716 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-zjw9d" Sep 10 12:18:15.552324 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:15.552288 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:15.552324 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:15.552310 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:15.552610 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:15.552288 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:18:15.552610 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:15.552432 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkj8" podUID="67095b3f-a8e2-494b-9f83-b4eb0b5634a2" Sep 10 12:18:15.552610 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:15.552501 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d7ls5" podUID="1b89f971-1fee-46bf-ad4c-e68e6b83b9a0" Sep 10 12:18:15.552739 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:15.552631 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vx2g" podUID="7169f42e-6be8-4806-bf45-c77b6e2bdfd1" Sep 10 12:18:15.734900 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:15.734832 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-zjw9d" Sep 10 12:18:15.735603 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:15.735575 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zjw9d" Sep 10 12:18:17.145289 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:17.145236 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/67095b3f-a8e2-494b-9f83-b4eb0b5634a2-original-pull-secret\") pod \"global-pull-secret-syncer-ljkj8\" (UID: \"67095b3f-a8e2-494b-9f83-b4eb0b5634a2\") " pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:17.145777 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:17.145411 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 12:18:17.145777 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:17.145505 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67095b3f-a8e2-494b-9f83-b4eb0b5634a2-original-pull-secret podName:67095b3f-a8e2-494b-9f83-b4eb0b5634a2 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:33.145487875 +0000 UTC m=+33.152804252 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/67095b3f-a8e2-494b-9f83-b4eb0b5634a2-original-pull-secret") pod "global-pull-secret-syncer-ljkj8" (UID: "67095b3f-a8e2-494b-9f83-b4eb0b5634a2") : object "kube-system"/"original-pull-secret" not registered Sep 10 12:18:17.245806 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:17.245767 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7169f42e-6be8-4806-bf45-c77b6e2bdfd1-metrics-certs\") pod \"network-metrics-daemon-7vx2g\" (UID: \"7169f42e-6be8-4806-bf45-c77b6e2bdfd1\") " pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:17.246041 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:17.245953 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:18:17.246041 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:17.246037 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7169f42e-6be8-4806-bf45-c77b6e2bdfd1-metrics-certs podName:7169f42e-6be8-4806-bf45-c77b6e2bdfd1 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:33.24601509 +0000 UTC m=+33.253331463 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7169f42e-6be8-4806-bf45-c77b6e2bdfd1-metrics-certs") pod "network-metrics-daemon-7vx2g" (UID: "7169f42e-6be8-4806-bf45-c77b6e2bdfd1") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:18:17.346892 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:17.346835 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-852dl\" (UniqueName: \"kubernetes.io/projected/1b89f971-1fee-46bf-ad4c-e68e6b83b9a0-kube-api-access-852dl\") pod \"network-check-target-d7ls5\" (UID: \"1b89f971-1fee-46bf-ad4c-e68e6b83b9a0\") " pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:18:17.347099 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:17.347001 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 12:18:17.347099 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:17.347031 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 12:18:17.347099 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:17.347043 2580 projected.go:196] Error preparing data for projected volume kube-api-access-852dl for pod openshift-network-diagnostics/network-check-target-d7ls5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:18:17.347225 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:17.347103 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b89f971-1fee-46bf-ad4c-e68e6b83b9a0-kube-api-access-852dl podName:1b89f971-1fee-46bf-ad4c-e68e6b83b9a0 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:33.34708708 +0000 UTC m=+33.354403441 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-852dl" (UniqueName: "kubernetes.io/projected/1b89f971-1fee-46bf-ad4c-e68e6b83b9a0-kube-api-access-852dl") pod "network-check-target-d7ls5" (UID: "1b89f971-1fee-46bf-ad4c-e68e6b83b9a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:18:17.551763 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:17.551722 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:17.551982 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:17.551741 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:17.551982 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:17.551878 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:18:17.551982 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:17.551871 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkj8" podUID="67095b3f-a8e2-494b-9f83-b4eb0b5634a2" Sep 10 12:18:17.552110 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:17.551984 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vx2g" podUID="7169f42e-6be8-4806-bf45-c77b6e2bdfd1" Sep 10 12:18:17.552110 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:17.552057 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d7ls5" podUID="1b89f971-1fee-46bf-ad4c-e68e6b83b9a0" Sep 10 12:18:19.552078 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:19.551741 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:19.552712 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:19.552207 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkj8" podUID="67095b3f-a8e2-494b-9f83-b4eb0b5634a2" Sep 10 12:18:19.552712 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:19.552650 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:19.552834 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:19.552757 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vx2g" podUID="7169f42e-6be8-4806-bf45-c77b6e2bdfd1" Sep 10 12:18:19.552834 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:19.552783 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:18:19.552956 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:19.552847 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d7ls5" podUID="1b89f971-1fee-46bf-ad4c-e68e6b83b9a0" Sep 10 12:18:19.746736 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:19.746691 2580 generic.go:356] "Generic (PLEG): container finished" podID="1d17b055-9484-48c5-bd77-90870f87d6ff" containerID="bcfd0dc8aa0714f9854aaf637d5126be11919bcfba532cbba2122066b0994c27" exitCode=0 Sep 10 12:18:19.746950 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:19.746776 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ft7lg" event={"ID":"1d17b055-9484-48c5-bd77-90870f87d6ff","Type":"ContainerDied","Data":"bcfd0dc8aa0714f9854aaf637d5126be11919bcfba532cbba2122066b0994c27"} Sep 10 12:18:19.749280 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:19.749250 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5g7lk" event={"ID":"0c94a7b9-9724-416f-8b9e-b98acd8c3c41","Type":"ContainerStarted","Data":"044a944a0fe5907583a6c0b7d10895291a86e8103f6d3d8266dd5f12eb0eece6"} Sep 10 12:18:19.752644 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:19.752526 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5mmp2" event={"ID":"08a3068f-f5f3-4086-8e21-444a4df81149","Type":"ContainerStarted","Data":"caa05641ad1f211cf48d55a8067e999305c21c89d65620350dfa07267b71ca6d"} Sep 10 12:18:19.794786 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:19.794712 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-5g7lk" podStartSLOduration=2.772620781 podStartE2EDuration="19.794688859s" totalImagesPullingTime="17.022068078s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:18:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:18:19.794645584 +0000 UTC m=+19.801961967" watchObservedRunningTime="2026-09-10 12:18:19.794688859 +0000 UTC m=+19.802005244" Sep 10 12:18:19.815710 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:19.815598 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5mmp2" podStartSLOduration=3.259292854 podStartE2EDuration="19.815579406s" totalImagesPullingTime="16.556286552s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:18:00 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:18:19.815290991 +0000 UTC m=+19.822607374" watchObservedRunningTime="2026-09-10 12:18:19.815579406 +0000 UTC m=+19.822895788" Sep 10 12:18:21.551458 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:21.551415 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:21.552106 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:21.551415 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:21.552106 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:21.551575 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkj8" podUID="67095b3f-a8e2-494b-9f83-b4eb0b5634a2" Sep 10 12:18:21.552106 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:21.551415 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:18:21.552106 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:21.551615 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vx2g" podUID="7169f42e-6be8-4806-bf45-c77b6e2bdfd1" Sep 10 12:18:21.552106 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:21.551691 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d7ls5" podUID="1b89f971-1fee-46bf-ad4c-e68e6b83b9a0" Sep 10 12:18:21.760062 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:21.760024 2580 generic.go:356] "Generic (PLEG): container finished" podID="1d17b055-9484-48c5-bd77-90870f87d6ff" containerID="6ae4028d77e624af409fabb20be1a4bc2dddf1f0c3502015e1e824289f21f859" exitCode=0 Sep 10 12:18:21.760267 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:21.760103 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ft7lg" event={"ID":"1d17b055-9484-48c5-bd77-90870f87d6ff","Type":"ContainerDied","Data":"6ae4028d77e624af409fabb20be1a4bc2dddf1f0c3502015e1e824289f21f859"} Sep 10 12:18:23.551390 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:23.551362 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:23.551390 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:23.551391 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:23.551806 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:23.551391 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:18:23.551806 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:23.551492 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkj8" podUID="67095b3f-a8e2-494b-9f83-b4eb0b5634a2" Sep 10 12:18:23.551806 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:23.551549 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vx2g" podUID="7169f42e-6be8-4806-bf45-c77b6e2bdfd1" Sep 10 12:18:23.551806 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:23.551598 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d7ls5" podUID="1b89f971-1fee-46bf-ad4c-e68e6b83b9a0" Sep 10 12:18:23.767221 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:23.767185 2580 generic.go:356] "Generic (PLEG): container finished" podID="1d17b055-9484-48c5-bd77-90870f87d6ff" containerID="c37279887258e6753dedc55544da68df20201f3578440b4a4baeb1ee2b1cc677" exitCode=0 Sep 10 12:18:23.767404 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:23.767245 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ft7lg" event={"ID":"1d17b055-9484-48c5-bd77-90870f87d6ff","Type":"ContainerDied","Data":"c37279887258e6753dedc55544da68df20201f3578440b4a4baeb1ee2b1cc677"} Sep 10 12:18:25.551620 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:25.551582 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:18:25.552185 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:25.551582 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:25.552185 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:25.551710 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d7ls5" podUID="1b89f971-1fee-46bf-ad4c-e68e6b83b9a0" Sep 10 12:18:25.552185 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:25.551582 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:25.552185 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:25.551771 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkj8" podUID="67095b3f-a8e2-494b-9f83-b4eb0b5634a2" Sep 10 12:18:25.552185 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:25.551890 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vx2g" podUID="7169f42e-6be8-4806-bf45-c77b6e2bdfd1" Sep 10 12:18:27.553066 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:27.552465 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:27.553066 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:27.552522 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:27.553066 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:27.552471 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:18:27.553066 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:27.552618 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkj8" podUID="67095b3f-a8e2-494b-9f83-b4eb0b5634a2" Sep 10 12:18:27.553066 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:27.552732 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d7ls5" podUID="1b89f971-1fee-46bf-ad4c-e68e6b83b9a0" Sep 10 12:18:27.553066 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:27.552842 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vx2g" podUID="7169f42e-6be8-4806-bf45-c77b6e2bdfd1" Sep 10 12:18:29.552307 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:29.552270 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:29.552839 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:29.552277 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:29.552839 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:29.552428 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vx2g" podUID="7169f42e-6be8-4806-bf45-c77b6e2bdfd1" Sep 10 12:18:29.552839 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:29.552270 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:18:29.552839 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:29.552486 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkj8" podUID="67095b3f-a8e2-494b-9f83-b4eb0b5634a2" Sep 10 12:18:29.552839 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:29.552534 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d7ls5" podUID="1b89f971-1fee-46bf-ad4c-e68e6b83b9a0" Sep 10 12:18:31.552421 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:31.552385 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:31.552935 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:31.552430 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:18:31.552935 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:31.552432 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:31.552935 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:31.552498 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkj8" podUID="67095b3f-a8e2-494b-9f83-b4eb0b5634a2" Sep 10 12:18:31.552935 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:31.552614 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vx2g" podUID="7169f42e-6be8-4806-bf45-c77b6e2bdfd1" Sep 10 12:18:31.552935 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:31.552682 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d7ls5" podUID="1b89f971-1fee-46bf-ad4c-e68e6b83b9a0" Sep 10 12:18:32.789831 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:32.789662 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" event={"ID":"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58","Type":"ContainerStarted","Data":"044791200fbf4b2a8447207810649c29b8d80ffaf47512befc87dafbe08a0945"} Sep 10 12:18:32.792430 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:32.792407 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ft7lg" event={"ID":"1d17b055-9484-48c5-bd77-90870f87d6ff","Type":"ContainerStarted","Data":"bb0c6dabc11919d0f35d4ab8f1c436d605863243b0e39c5ab3c16495b53960fb"} Sep 10 12:18:33.165539 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:33.165515 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/67095b3f-a8e2-494b-9f83-b4eb0b5634a2-original-pull-secret\") pod \"global-pull-secret-syncer-ljkj8\" (UID: \"67095b3f-a8e2-494b-9f83-b4eb0b5634a2\") " pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:33.165697 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:33.165671 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 12:18:33.165820 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:33.165734 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67095b3f-a8e2-494b-9f83-b4eb0b5634a2-original-pull-secret podName:67095b3f-a8e2-494b-9f83-b4eb0b5634a2 nodeName:}" failed. No retries permitted until 2026-09-10 12:19:05.165719967 +0000 UTC m=+65.173036332 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/67095b3f-a8e2-494b-9f83-b4eb0b5634a2-original-pull-secret") pod "global-pull-secret-syncer-ljkj8" (UID: "67095b3f-a8e2-494b-9f83-b4eb0b5634a2") : object "kube-system"/"original-pull-secret" not registered Sep 10 12:18:33.266756 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:33.266714 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7169f42e-6be8-4806-bf45-c77b6e2bdfd1-metrics-certs\") pod \"network-metrics-daemon-7vx2g\" (UID: \"7169f42e-6be8-4806-bf45-c77b6e2bdfd1\") " pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:33.266924 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:33.266846 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:18:33.266924 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:33.266914 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7169f42e-6be8-4806-bf45-c77b6e2bdfd1-metrics-certs podName:7169f42e-6be8-4806-bf45-c77b6e2bdfd1 nodeName:}" failed. No retries permitted until 2026-09-10 12:19:05.266897126 +0000 UTC m=+65.274213512 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7169f42e-6be8-4806-bf45-c77b6e2bdfd1-metrics-certs") pod "network-metrics-daemon-7vx2g" (UID: "7169f42e-6be8-4806-bf45-c77b6e2bdfd1") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:18:33.368080 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:33.368009 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-852dl\" (UniqueName: \"kubernetes.io/projected/1b89f971-1fee-46bf-ad4c-e68e6b83b9a0-kube-api-access-852dl\") pod \"network-check-target-d7ls5\" (UID: \"1b89f971-1fee-46bf-ad4c-e68e6b83b9a0\") " pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:18:33.368207 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:33.368156 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 12:18:33.368207 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:33.368170 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 12:18:33.368207 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:33.368178 2580 projected.go:196] Error preparing data for projected volume kube-api-access-852dl for pod openshift-network-diagnostics/network-check-target-d7ls5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:18:33.368304 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:33.368223 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b89f971-1fee-46bf-ad4c-e68e6b83b9a0-kube-api-access-852dl podName:1b89f971-1fee-46bf-ad4c-e68e6b83b9a0 nodeName:}" failed. No retries permitted until 2026-09-10 12:19:05.368211555 +0000 UTC m=+65.375527916 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-852dl" (UniqueName: "kubernetes.io/projected/1b89f971-1fee-46bf-ad4c-e68e6b83b9a0-kube-api-access-852dl") pod "network-check-target-d7ls5" (UID: "1b89f971-1fee-46bf-ad4c-e68e6b83b9a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:18:33.552045 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:33.552012 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:18:33.552194 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:33.552112 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d7ls5" podUID="1b89f971-1fee-46bf-ad4c-e68e6b83b9a0" Sep 10 12:18:33.552194 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:33.552025 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:33.552194 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:33.552160 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:33.552305 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:33.552234 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vx2g" podUID="7169f42e-6be8-4806-bf45-c77b6e2bdfd1" Sep 10 12:18:33.552338 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:33.552300 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkj8" podUID="67095b3f-a8e2-494b-9f83-b4eb0b5634a2" Sep 10 12:18:33.797148 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:33.797120 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tfcmz_e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58/ovn-acl-logging/0.log" Sep 10 12:18:33.797588 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:33.797449 2580 generic.go:356] "Generic (PLEG): container finished" podID="e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58" containerID="55a0d72daf993164477d0dc49db85e8c6678aaa5e6b4f58cf2a50f4771d4202b" exitCode=1 Sep 10 12:18:33.797588 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:33.797525 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" event={"ID":"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58","Type":"ContainerStarted","Data":"dae113a1c355a6595e5446a62f7976397ddfa9efdcb3ef75d01f4705db24ffd3"} Sep 10 12:18:33.797664 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:33.797591 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" event={"ID":"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58","Type":"ContainerStarted","Data":"0bd0e80b850448f64efbb033cc4f0376341c3e58929f5d0d7a43b6f3e8c549cc"} Sep 10 12:18:33.797664 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:33.797630 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" event={"ID":"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58","Type":"ContainerStarted","Data":"18ddfe95b5fd680714ddd4df891ed8e6757147d52e9789e1f32261ac7cfadb14"} Sep 10 12:18:33.797664 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:33.797657 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" event={"ID":"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58","Type":"ContainerStarted","Data":"fdff9e3f3eba5fbea89b61832eb60999d009520282f1a975517faad03293f4f4"} Sep 10 12:18:33.797794 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:33.797701 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" event={"ID":"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58","Type":"ContainerDied","Data":"55a0d72daf993164477d0dc49db85e8c6678aaa5e6b4f58cf2a50f4771d4202b"} Sep 10 12:18:33.800248 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:33.800227 2580 generic.go:356] "Generic (PLEG): container finished" podID="1d17b055-9484-48c5-bd77-90870f87d6ff" containerID="bb0c6dabc11919d0f35d4ab8f1c436d605863243b0e39c5ab3c16495b53960fb" exitCode=0 Sep 10 12:18:33.800344 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:33.800256 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ft7lg" event={"ID":"1d17b055-9484-48c5-bd77-90870f87d6ff","Type":"ContainerDied","Data":"bb0c6dabc11919d0f35d4ab8f1c436d605863243b0e39c5ab3c16495b53960fb"} Sep 10 12:18:34.805470 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:34.805432 2580 generic.go:356] "Generic (PLEG): container finished" podID="1d17b055-9484-48c5-bd77-90870f87d6ff" containerID="dac7b39bfbcf2c34f661db31c589209fb50aee812d3a738599dedee94c6a2f90" exitCode=0 Sep 10 12:18:34.805888 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:34.805501 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ft7lg" event={"ID":"1d17b055-9484-48c5-bd77-90870f87d6ff","Type":"ContainerDied","Data":"dac7b39bfbcf2c34f661db31c589209fb50aee812d3a738599dedee94c6a2f90"} Sep 10 12:18:35.551447 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:35.551372 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:18:35.551600 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:35.551372 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:35.551600 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:35.551471 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d7ls5" podUID="1b89f971-1fee-46bf-ad4c-e68e6b83b9a0" Sep 10 12:18:35.551600 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:35.551372 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:35.551600 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:35.551563 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vx2g" podUID="7169f42e-6be8-4806-bf45-c77b6e2bdfd1" Sep 10 12:18:35.551781 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:35.551647 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkj8" podUID="67095b3f-a8e2-494b-9f83-b4eb0b5634a2" Sep 10 12:18:35.813262 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:35.813182 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tfcmz_e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58/ovn-acl-logging/0.log" Sep 10 12:18:35.813676 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:35.813626 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" event={"ID":"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58","Type":"ContainerStarted","Data":"d5d42e04a5d768b402b3044789b5188eaadfca8ef61961eed3e282e8369bc145"} Sep 10 12:18:35.816810 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:35.816787 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ft7lg" event={"ID":"1d17b055-9484-48c5-bd77-90870f87d6ff","Type":"ContainerStarted","Data":"8dfca573ade09387be64a3d7e14e945cdd283f016ea740553112e4b241e77dad"} Sep 10 12:18:35.844355 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:35.844313 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ft7lg" podStartSLOduration=7.880104093 podStartE2EDuration="35.84429894s" totalImagesPullingTime="27.964194847s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:18:00 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:18:35.843907669 +0000 UTC m=+35.851224051" watchObservedRunningTime="2026-09-10 12:18:35.84429894 +0000 UTC m=+35.851615322" Sep 10 12:18:37.552387 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:37.551816 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:37.552387 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:37.551818 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:18:37.552387 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:37.552084 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vx2g" podUID="7169f42e-6be8-4806-bf45-c77b6e2bdfd1" Sep 10 12:18:37.552387 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:37.551886 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:37.552387 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:37.552156 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d7ls5" podUID="1b89f971-1fee-46bf-ad4c-e68e6b83b9a0" Sep 10 12:18:37.552387 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:37.552239 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkj8" podUID="67095b3f-a8e2-494b-9f83-b4eb0b5634a2" Sep 10 12:18:37.824102 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:37.824032 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tfcmz_e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58/ovn-acl-logging/0.log" Sep 10 12:18:37.824432 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:37.824399 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" event={"ID":"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58","Type":"ContainerStarted","Data":"0f596a2b0b7773e2f23bb8e3a9403f0a46baa7942a057a1eb231c985f76c7f2f"} Sep 10 12:18:37.824693 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:37.824648 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:37.824869 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:37.824834 2580 scope.go:117] "RemoveContainer" containerID="55a0d72daf993164477d0dc49db85e8c6678aaa5e6b4f58cf2a50f4771d4202b" Sep 10 12:18:37.840205 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:37.840181 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:38.829527 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:38.829503 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tfcmz_e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58/ovn-acl-logging/0.log" Sep 10 12:18:38.830146 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:38.829840 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" event={"ID":"e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58","Type":"ContainerStarted","Data":"5cc9f26c7445c20fc9538d795dac668cf845c77afed47e920fb3969386fbdc37"} Sep 10 12:18:38.830146 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:38.830028 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:38.830146 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:38.830103 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:38.844419 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:38.844398 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:18:38.871643 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:38.871605 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" podStartSLOduration=8.102687708 podStartE2EDuration="38.871592362s" totalImagesPullingTime="30.768904654s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:18:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=2 observedRunningTime="2026-09-10 12:18:38.870208953 +0000 UTC m=+38.877525337" watchObservedRunningTime="2026-09-10 12:18:38.871592362 +0000 UTC m=+38.878908744" Sep 10 12:18:38.903330 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:38.903311 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pkt25_2a76158d-6412-48e2-a091-51cc34552ce2/dns-node-resolver/0.log" Sep 10 12:18:39.486028 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:39.485994 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ljkj8"] Sep 10 12:18:39.486269 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:39.486122 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:39.486269 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:39.486226 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkj8" podUID="67095b3f-a8e2-494b-9f83-b4eb0b5634a2" Sep 10 12:18:39.486626 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:39.486603 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-d7ls5"] Sep 10 12:18:39.486732 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:39.486687 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:18:39.486794 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:39.486765 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d7ls5" podUID="1b89f971-1fee-46bf-ad4c-e68e6b83b9a0" Sep 10 12:18:39.487121 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:39.487100 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7vx2g"] Sep 10 12:18:39.487190 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:39.487180 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:39.487270 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:39.487256 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vx2g" podUID="7169f42e-6be8-4806-bf45-c77b6e2bdfd1" Sep 10 12:18:39.683716 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:39.683691 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-68mrh_e6b25f25-3e5d-45f3-8c46-7884a14b981f/node-ca/0.log" Sep 10 12:18:40.552575 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:40.552543 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:40.553112 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:40.552624 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkj8" podUID="67095b3f-a8e2-494b-9f83-b4eb0b5634a2" Sep 10 12:18:41.551933 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:41.551758 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:18:41.552081 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:41.551794 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:41.552081 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:41.552009 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d7ls5" podUID="1b89f971-1fee-46bf-ad4c-e68e6b83b9a0" Sep 10 12:18:41.552202 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:41.552121 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vx2g" podUID="7169f42e-6be8-4806-bf45-c77b6e2bdfd1" Sep 10 12:18:42.551680 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:42.551643 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:42.552158 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:42.551750 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkj8" podUID="67095b3f-a8e2-494b-9f83-b4eb0b5634a2" Sep 10 12:18:43.552060 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:43.552032 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:18:43.552405 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:43.552126 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d7ls5" podUID="1b89f971-1fee-46bf-ad4c-e68e6b83b9a0" Sep 10 12:18:43.552405 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:43.552204 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:43.552405 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:43.552304 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vx2g" podUID="7169f42e-6be8-4806-bf45-c77b6e2bdfd1" Sep 10 12:18:44.551924 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:44.551886 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:44.552116 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:44.552017 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkj8" podUID="67095b3f-a8e2-494b-9f83-b4eb0b5634a2" Sep 10 12:18:45.551544 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:45.551520 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:45.551685 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:45.551519 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:18:45.551685 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:45.551626 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vx2g" podUID="7169f42e-6be8-4806-bf45-c77b6e2bdfd1" Sep 10 12:18:45.551787 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:45.551717 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d7ls5" podUID="1b89f971-1fee-46bf-ad4c-e68e6b83b9a0" Sep 10 12:18:46.552265 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:46.552231 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:46.552732 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:46.552352 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkj8" podUID="67095b3f-a8e2-494b-9f83-b4eb0b5634a2" Sep 10 12:18:47.552371 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:47.552337 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:18:47.552806 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:47.552340 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:47.552806 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:47.552460 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d7ls5" podUID="1b89f971-1fee-46bf-ad4c-e68e6b83b9a0" Sep 10 12:18:47.552806 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:47.552509 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vx2g" podUID="7169f42e-6be8-4806-bf45-c77b6e2bdfd1" Sep 10 12:18:48.552036 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:48.552006 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:48.552199 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:48.552111 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkj8" podUID="67095b3f-a8e2-494b-9f83-b4eb0b5634a2" Sep 10 12:18:49.551431 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:49.551400 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:49.551764 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:49.551502 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vx2g" podUID="7169f42e-6be8-4806-bf45-c77b6e2bdfd1" Sep 10 12:18:49.551764 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:49.551554 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:18:49.551764 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:49.551636 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d7ls5" podUID="1b89f971-1fee-46bf-ad4c-e68e6b83b9a0" Sep 10 12:18:50.552453 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:50.552424 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:50.552816 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:50.552505 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkj8" podUID="67095b3f-a8e2-494b-9f83-b4eb0b5634a2" Sep 10 12:18:51.552269 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:51.552238 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:18:51.552429 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:51.552238 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:51.552429 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:51.552345 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d7ls5" podUID="1b89f971-1fee-46bf-ad4c-e68e6b83b9a0" Sep 10 12:18:51.552429 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:51.552413 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vx2g" podUID="7169f42e-6be8-4806-bf45-c77b6e2bdfd1" Sep 10 12:18:52.551469 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:52.551437 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:52.551625 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:52.551572 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkj8" podUID="67095b3f-a8e2-494b-9f83-b4eb0b5634a2" Sep 10 12:18:53.552164 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:53.552131 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:53.552572 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:53.552240 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vx2g" podUID="7169f42e-6be8-4806-bf45-c77b6e2bdfd1" Sep 10 12:18:53.552572 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:53.552293 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:18:53.552572 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:53.552378 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d7ls5" podUID="1b89f971-1fee-46bf-ad4c-e68e6b83b9a0" Sep 10 12:18:54.551945 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:54.551912 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:54.552096 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:54.552042 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkj8" podUID="67095b3f-a8e2-494b-9f83-b4eb0b5634a2" Sep 10 12:18:55.552028 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:55.551998 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:18:55.552405 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:55.552040 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:55.552405 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:55.552113 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d7ls5" podUID="1b89f971-1fee-46bf-ad4c-e68e6b83b9a0" Sep 10 12:18:55.552405 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:55.552238 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vx2g" podUID="7169f42e-6be8-4806-bf45-c77b6e2bdfd1" Sep 10 12:18:56.551427 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:56.551399 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:56.551590 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:56.551495 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkj8" podUID="67095b3f-a8e2-494b-9f83-b4eb0b5634a2" Sep 10 12:18:57.551921 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:57.551896 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:18:57.551921 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:57.551916 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:57.552263 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:57.551986 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d7ls5" podUID="1b89f971-1fee-46bf-ad4c-e68e6b83b9a0" Sep 10 12:18:57.552263 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:57.552105 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vx2g" podUID="7169f42e-6be8-4806-bf45-c77b6e2bdfd1" Sep 10 12:18:58.552458 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:58.552419 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:18:58.552914 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:58.552552 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkj8" podUID="67095b3f-a8e2-494b-9f83-b4eb0b5634a2" Sep 10 12:18:59.552107 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:59.552074 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:18:59.552251 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:18:59.552085 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:18:59.552286 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:59.552169 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d7ls5" podUID="1b89f971-1fee-46bf-ad4c-e68e6b83b9a0" Sep 10 12:18:59.552286 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:18:59.552262 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vx2g" podUID="7169f42e-6be8-4806-bf45-c77b6e2bdfd1" Sep 10 12:19:00.552506 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:00.552477 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:19:00.552937 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:19:00.552558 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkj8" podUID="67095b3f-a8e2-494b-9f83-b4eb0b5634a2" Sep 10 12:19:01.551408 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:01.551376 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:19:01.551408 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:01.551394 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:19:01.551604 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:19:01.551476 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d7ls5" podUID="1b89f971-1fee-46bf-ad4c-e68e6b83b9a0" Sep 10 12:19:01.551641 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:19:01.551599 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vx2g" podUID="7169f42e-6be8-4806-bf45-c77b6e2bdfd1" Sep 10 12:19:02.554454 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:02.554424 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:19:02.554812 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:19:02.554520 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ljkj8" podUID="67095b3f-a8e2-494b-9f83-b4eb0b5634a2" Sep 10 12:19:03.551974 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.551897 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:19:03.551974 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.551925 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:19:03.552141 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:19:03.552002 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vx2g" podUID="7169f42e-6be8-4806-bf45-c77b6e2bdfd1" Sep 10 12:19:03.552141 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:19:03.552108 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d7ls5" podUID="1b89f971-1fee-46bf-ad4c-e68e6b83b9a0" Sep 10 12:19:03.741276 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.741252 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-221.ec2.internal" event="NodeReady" Sep 10 12:19:03.741616 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.741370 2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 10 12:19:03.801941 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.801912 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-sjwmh"] Sep 10 12:19:03.824318 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.824249 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-t5mdr"] Sep 10 12:19:03.824448 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.824321 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sjwmh" Sep 10 12:19:03.829441 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.829362 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lhd7h\"" Sep 10 12:19:03.829441 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.829378 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 10 12:19:03.829441 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.829399 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 10 12:19:03.847750 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.847726 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-sjwmh"] Sep 10 12:19:03.847883 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.847758 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-t5mdr"] Sep 10 12:19:03.847883 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.847836 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-t5mdr" Sep 10 12:19:03.851107 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.851085 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 10 12:19:03.851372 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.851355 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 10 12:19:03.851372 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.851364 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 10 12:19:03.851535 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.851459 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zcn9n\"" Sep 10 12:19:03.851535 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.851472 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 10 12:19:03.887810 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.887788 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42e548ce-35b0-4117-952f-c0b6bec98b14-tmp-dir\") pod \"dns-default-sjwmh\" (UID: \"42e548ce-35b0-4117-952f-c0b6bec98b14\") " pod="openshift-dns/dns-default-sjwmh" Sep 10 12:19:03.887961 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.887837 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hd6s5\" (UniqueName: \"kubernetes.io/projected/42e548ce-35b0-4117-952f-c0b6bec98b14-kube-api-access-hd6s5\") pod \"dns-default-sjwmh\" (UID: \"42e548ce-35b0-4117-952f-c0b6bec98b14\") " pod="openshift-dns/dns-default-sjwmh" Sep 10 12:19:03.887961 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.887871 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5b1d6ed3-daa3-4642-84d3-dd8cfe53e110-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t5mdr\" (UID: \"5b1d6ed3-daa3-4642-84d3-dd8cfe53e110\") " pod="openshift-insights/insights-runtime-extractor-t5mdr" Sep 10 12:19:03.887961 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.887920 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b1d6ed3-daa3-4642-84d3-dd8cfe53e110-data-volume\") pod \"insights-runtime-extractor-t5mdr\" (UID: \"5b1d6ed3-daa3-4642-84d3-dd8cfe53e110\") " pod="openshift-insights/insights-runtime-extractor-t5mdr" Sep 10 12:19:03.887961 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.887953 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82cwd\" (UniqueName: \"kubernetes.io/projected/5b1d6ed3-daa3-4642-84d3-dd8cfe53e110-kube-api-access-82cwd\") pod \"insights-runtime-extractor-t5mdr\" (UID: \"5b1d6ed3-daa3-4642-84d3-dd8cfe53e110\") " pod="openshift-insights/insights-runtime-extractor-t5mdr" Sep 10 12:19:03.888091 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.887982 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5b1d6ed3-daa3-4642-84d3-dd8cfe53e110-crio-socket\") pod \"insights-runtime-extractor-t5mdr\" (UID: \"5b1d6ed3-daa3-4642-84d3-dd8cfe53e110\") " pod="openshift-insights/insights-runtime-extractor-t5mdr" Sep 10 12:19:03.888091 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.888015 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/42e548ce-35b0-4117-952f-c0b6bec98b14-config-volume\") pod \"dns-default-sjwmh\" (UID: \"42e548ce-35b0-4117-952f-c0b6bec98b14\") " pod="openshift-dns/dns-default-sjwmh" Sep 10 12:19:03.888091 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.888035 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/42e548ce-35b0-4117-952f-c0b6bec98b14-metrics-tls\") pod \"dns-default-sjwmh\" (UID: \"42e548ce-35b0-4117-952f-c0b6bec98b14\") " pod="openshift-dns/dns-default-sjwmh" Sep 10 12:19:03.888091 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.888058 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5b1d6ed3-daa3-4642-84d3-dd8cfe53e110-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t5mdr\" (UID: \"5b1d6ed3-daa3-4642-84d3-dd8cfe53e110\") " pod="openshift-insights/insights-runtime-extractor-t5mdr" Sep 10 12:19:03.900154 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.900128 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-2f4lc"] Sep 10 12:19:03.921713 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.921682 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2f4lc"] Sep 10 12:19:03.921832 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.921792 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2f4lc" Sep 10 12:19:03.925318 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.925299 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 10 12:19:03.925536 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.925317 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 10 12:19:03.925659 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.925540 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 10 12:19:03.925762 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.925747 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xh5g6\"" Sep 10 12:19:03.988967 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.988934 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5b1d6ed3-daa3-4642-84d3-dd8cfe53e110-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t5mdr\" (UID: \"5b1d6ed3-daa3-4642-84d3-dd8cfe53e110\") " pod="openshift-insights/insights-runtime-extractor-t5mdr" Sep 10 12:19:03.989138 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.988973 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b1d6ed3-daa3-4642-84d3-dd8cfe53e110-data-volume\") pod \"insights-runtime-extractor-t5mdr\" (UID: \"5b1d6ed3-daa3-4642-84d3-dd8cfe53e110\") " pod="openshift-insights/insights-runtime-extractor-t5mdr" Sep 10 12:19:03.989138 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.989002 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82cwd\" (UniqueName: \"kubernetes.io/projected/5b1d6ed3-daa3-4642-84d3-dd8cfe53e110-kube-api-access-82cwd\") pod \"insights-runtime-extractor-t5mdr\" (UID: \"5b1d6ed3-daa3-4642-84d3-dd8cfe53e110\") " pod="openshift-insights/insights-runtime-extractor-t5mdr" Sep 10 12:19:03.989138 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.989032 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5b1d6ed3-daa3-4642-84d3-dd8cfe53e110-crio-socket\") pod \"insights-runtime-extractor-t5mdr\" (UID: \"5b1d6ed3-daa3-4642-84d3-dd8cfe53e110\") " pod="openshift-insights/insights-runtime-extractor-t5mdr" Sep 10 12:19:03.989138 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.989071 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/42e548ce-35b0-4117-952f-c0b6bec98b14-config-volume\") pod \"dns-default-sjwmh\" (UID: \"42e548ce-35b0-4117-952f-c0b6bec98b14\") " pod="openshift-dns/dns-default-sjwmh" Sep 10 12:19:03.989138 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.989095 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/42e548ce-35b0-4117-952f-c0b6bec98b14-metrics-tls\") pod \"dns-default-sjwmh\" (UID: \"42e548ce-35b0-4117-952f-c0b6bec98b14\") " pod="openshift-dns/dns-default-sjwmh" Sep 10 12:19:03.989138 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.989118 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5b1d6ed3-daa3-4642-84d3-dd8cfe53e110-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t5mdr\" (UID: \"5b1d6ed3-daa3-4642-84d3-dd8cfe53e110\") " pod="openshift-insights/insights-runtime-extractor-t5mdr" Sep 10 12:19:03.989436 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.989162 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42e548ce-35b0-4117-952f-c0b6bec98b14-tmp-dir\") pod \"dns-default-sjwmh\" (UID: \"42e548ce-35b0-4117-952f-c0b6bec98b14\") " pod="openshift-dns/dns-default-sjwmh" Sep 10 12:19:03.989436 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.989213 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg79x\" (UniqueName: \"kubernetes.io/projected/e09b8c21-6ece-4589-8fa2-dfa2c0c0afe4-kube-api-access-mg79x\") pod \"ingress-canary-2f4lc\" (UID: \"e09b8c21-6ece-4589-8fa2-dfa2c0c0afe4\") " pod="openshift-ingress-canary/ingress-canary-2f4lc" Sep 10 12:19:03.989436 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.989210 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5b1d6ed3-daa3-4642-84d3-dd8cfe53e110-crio-socket\") pod \"insights-runtime-extractor-t5mdr\" (UID: \"5b1d6ed3-daa3-4642-84d3-dd8cfe53e110\") " pod="openshift-insights/insights-runtime-extractor-t5mdr" Sep 10 12:19:03.989436 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.989242 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e09b8c21-6ece-4589-8fa2-dfa2c0c0afe4-cert\") pod \"ingress-canary-2f4lc\" (UID: \"e09b8c21-6ece-4589-8fa2-dfa2c0c0afe4\") " pod="openshift-ingress-canary/ingress-canary-2f4lc" Sep 10 12:19:03.989436 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.989281 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hd6s5\" (UniqueName: \"kubernetes.io/projected/42e548ce-35b0-4117-952f-c0b6bec98b14-kube-api-access-hd6s5\") pod \"dns-default-sjwmh\" (UID: \"42e548ce-35b0-4117-952f-c0b6bec98b14\") " pod="openshift-dns/dns-default-sjwmh" Sep 10 12:19:03.989436 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.989356 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b1d6ed3-daa3-4642-84d3-dd8cfe53e110-data-volume\") pod \"insights-runtime-extractor-t5mdr\" (UID: \"5b1d6ed3-daa3-4642-84d3-dd8cfe53e110\") " pod="openshift-insights/insights-runtime-extractor-t5mdr" Sep 10 12:19:03.989702 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.989628 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42e548ce-35b0-4117-952f-c0b6bec98b14-tmp-dir\") pod \"dns-default-sjwmh\" (UID: \"42e548ce-35b0-4117-952f-c0b6bec98b14\") " pod="openshift-dns/dns-default-sjwmh" Sep 10 12:19:03.989702 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.989696 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5b1d6ed3-daa3-4642-84d3-dd8cfe53e110-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t5mdr\" (UID: \"5b1d6ed3-daa3-4642-84d3-dd8cfe53e110\") " pod="openshift-insights/insights-runtime-extractor-t5mdr" Sep 10 12:19:03.989800 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.989710 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/42e548ce-35b0-4117-952f-c0b6bec98b14-config-volume\") pod \"dns-default-sjwmh\" (UID: \"42e548ce-35b0-4117-952f-c0b6bec98b14\") " pod="openshift-dns/dns-default-sjwmh" Sep 10 12:19:03.992328 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.992306 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5b1d6ed3-daa3-4642-84d3-dd8cfe53e110-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t5mdr\" (UID: \"5b1d6ed3-daa3-4642-84d3-dd8cfe53e110\") " pod="openshift-insights/insights-runtime-extractor-t5mdr" Sep 10 12:19:03.992415 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.992306 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/42e548ce-35b0-4117-952f-c0b6bec98b14-metrics-tls\") pod \"dns-default-sjwmh\" (UID: \"42e548ce-35b0-4117-952f-c0b6bec98b14\") " pod="openshift-dns/dns-default-sjwmh" Sep 10 12:19:03.999769 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:03.999746 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hd6s5\" (UniqueName: \"kubernetes.io/projected/42e548ce-35b0-4117-952f-c0b6bec98b14-kube-api-access-hd6s5\") pod \"dns-default-sjwmh\" (UID: \"42e548ce-35b0-4117-952f-c0b6bec98b14\") " pod="openshift-dns/dns-default-sjwmh" Sep 10 12:19:04.001178 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:04.001159 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82cwd\" (UniqueName: \"kubernetes.io/projected/5b1d6ed3-daa3-4642-84d3-dd8cfe53e110-kube-api-access-82cwd\") pod \"insights-runtime-extractor-t5mdr\" (UID: \"5b1d6ed3-daa3-4642-84d3-dd8cfe53e110\") " pod="openshift-insights/insights-runtime-extractor-t5mdr" Sep 10 12:19:04.090444 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:04.090371 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg79x\" (UniqueName: \"kubernetes.io/projected/e09b8c21-6ece-4589-8fa2-dfa2c0c0afe4-kube-api-access-mg79x\") pod \"ingress-canary-2f4lc\" (UID: \"e09b8c21-6ece-4589-8fa2-dfa2c0c0afe4\") " pod="openshift-ingress-canary/ingress-canary-2f4lc" Sep 10 12:19:04.090444 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:04.090405 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e09b8c21-6ece-4589-8fa2-dfa2c0c0afe4-cert\") pod \"ingress-canary-2f4lc\" (UID: \"e09b8c21-6ece-4589-8fa2-dfa2c0c0afe4\") " pod="openshift-ingress-canary/ingress-canary-2f4lc" Sep 10 12:19:04.092621 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:04.092603 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e09b8c21-6ece-4589-8fa2-dfa2c0c0afe4-cert\") pod \"ingress-canary-2f4lc\" (UID: \"e09b8c21-6ece-4589-8fa2-dfa2c0c0afe4\") " pod="openshift-ingress-canary/ingress-canary-2f4lc" Sep 10 12:19:04.099443 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:04.099418 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg79x\" (UniqueName: \"kubernetes.io/projected/e09b8c21-6ece-4589-8fa2-dfa2c0c0afe4-kube-api-access-mg79x\") pod \"ingress-canary-2f4lc\" (UID: \"e09b8c21-6ece-4589-8fa2-dfa2c0c0afe4\") " pod="openshift-ingress-canary/ingress-canary-2f4lc" Sep 10 12:19:04.134303 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:04.134279 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sjwmh" Sep 10 12:19:04.155980 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:04.155958 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-t5mdr" Sep 10 12:19:04.229708 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:04.229671 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2f4lc" Sep 10 12:19:04.464199 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:04.464169 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-t5mdr"] Sep 10 12:19:04.467050 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:04.467030 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-sjwmh"] Sep 10 12:19:04.468231 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:19:04.468209 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b1d6ed3_daa3_4642_84d3_dd8cfe53e110.slice/crio-8a528a29279cb28dec6027b1784f83a26ca6c1b41de170899fe9a540a5847be3 WatchSource:0}: Error finding container 8a528a29279cb28dec6027b1784f83a26ca6c1b41de170899fe9a540a5847be3: Status 404 returned error can't find the container with id 8a528a29279cb28dec6027b1784f83a26ca6c1b41de170899fe9a540a5847be3 Sep 10 12:19:04.469745 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:19:04.469723 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42e548ce_35b0_4117_952f_c0b6bec98b14.slice/crio-1c01c7c048d9c729b64d3f6a239ff8360294a64853447c49ff3bdac2c4d6aea1 WatchSource:0}: Error finding container 1c01c7c048d9c729b64d3f6a239ff8360294a64853447c49ff3bdac2c4d6aea1: Status 404 returned error can't find the container with id 1c01c7c048d9c729b64d3f6a239ff8360294a64853447c49ff3bdac2c4d6aea1 Sep 10 12:19:04.517575 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:04.517549 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2f4lc"] Sep 10 12:19:04.526743 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:19:04.526710 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode09b8c21_6ece_4589_8fa2_dfa2c0c0afe4.slice/crio-37a7a1064d2e0509a06823f827768ccba33239b6044ca95fa429776973a61bc7 WatchSource:0}: Error finding container 37a7a1064d2e0509a06823f827768ccba33239b6044ca95fa429776973a61bc7: Status 404 returned error can't find the container with id 37a7a1064d2e0509a06823f827768ccba33239b6044ca95fa429776973a61bc7 Sep 10 12:19:04.555453 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:04.555430 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:19:04.558453 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:04.558430 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 10 12:19:04.886341 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:04.886261 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2f4lc" event={"ID":"e09b8c21-6ece-4589-8fa2-dfa2c0c0afe4","Type":"ContainerStarted","Data":"37a7a1064d2e0509a06823f827768ccba33239b6044ca95fa429776973a61bc7"} Sep 10 12:19:04.887634 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:04.887610 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t5mdr" event={"ID":"5b1d6ed3-daa3-4642-84d3-dd8cfe53e110","Type":"ContainerStarted","Data":"5ac544aafb5e0fc86b7728640202864bed921f34ea11669699f9b992a194b998"} Sep 10 12:19:04.887786 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:04.887653 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t5mdr" event={"ID":"5b1d6ed3-daa3-4642-84d3-dd8cfe53e110","Type":"ContainerStarted","Data":"8a528a29279cb28dec6027b1784f83a26ca6c1b41de170899fe9a540a5847be3"} Sep 10 12:19:04.888610 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:04.888592 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sjwmh" event={"ID":"42e548ce-35b0-4117-952f-c0b6bec98b14","Type":"ContainerStarted","Data":"1c01c7c048d9c729b64d3f6a239ff8360294a64853447c49ff3bdac2c4d6aea1"} Sep 10 12:19:05.200183 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:05.200137 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/67095b3f-a8e2-494b-9f83-b4eb0b5634a2-original-pull-secret\") pod \"global-pull-secret-syncer-ljkj8\" (UID: \"67095b3f-a8e2-494b-9f83-b4eb0b5634a2\") " pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:19:05.202947 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:05.202917 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/67095b3f-a8e2-494b-9f83-b4eb0b5634a2-original-pull-secret\") pod \"global-pull-secret-syncer-ljkj8\" (UID: \"67095b3f-a8e2-494b-9f83-b4eb0b5634a2\") " pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:19:05.300818 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:05.300780 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7169f42e-6be8-4806-bf45-c77b6e2bdfd1-metrics-certs\") pod \"network-metrics-daemon-7vx2g\" (UID: \"7169f42e-6be8-4806-bf45-c77b6e2bdfd1\") " pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:19:05.301010 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:19:05.300943 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:19:05.301076 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:19:05.301018 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7169f42e-6be8-4806-bf45-c77b6e2bdfd1-metrics-certs podName:7169f42e-6be8-4806-bf45-c77b6e2bdfd1 nodeName:}" failed. No retries permitted until 2026-09-10 12:20:09.300998846 +0000 UTC m=+129.308315222 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7169f42e-6be8-4806-bf45-c77b6e2bdfd1-metrics-certs") pod "network-metrics-daemon-7vx2g" (UID: "7169f42e-6be8-4806-bf45-c77b6e2bdfd1") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:19:05.401698 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:05.401658 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-852dl\" (UniqueName: \"kubernetes.io/projected/1b89f971-1fee-46bf-ad4c-e68e6b83b9a0-kube-api-access-852dl\") pod \"network-check-target-d7ls5\" (UID: \"1b89f971-1fee-46bf-ad4c-e68e6b83b9a0\") " pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:19:05.401891 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:19:05.401808 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 12:19:05.401891 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:19:05.401835 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 12:19:05.402014 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:19:05.401851 2580 projected.go:196] Error preparing data for projected volume kube-api-access-852dl for pod openshift-network-diagnostics/network-check-target-d7ls5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:19:05.402067 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:19:05.402020 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b89f971-1fee-46bf-ad4c-e68e6b83b9a0-kube-api-access-852dl podName:1b89f971-1fee-46bf-ad4c-e68e6b83b9a0 nodeName:}" failed. No retries permitted until 2026-09-10 12:20:09.402000592 +0000 UTC m=+129.409316961 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "kube-api-access-852dl" (UniqueName: "kubernetes.io/projected/1b89f971-1fee-46bf-ad4c-e68e6b83b9a0-kube-api-access-852dl") pod "network-check-target-d7ls5" (UID: "1b89f971-1fee-46bf-ad4c-e68e6b83b9a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:19:05.463477 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:05.463400 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ljkj8" Sep 10 12:19:05.551946 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:05.551907 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:19:05.552478 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:05.551911 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:19:05.556700 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:05.556447 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 12:19:05.556700 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:05.556490 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 10 12:19:05.556700 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:05.556538 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cfjr5\"" Sep 10 12:19:05.556700 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:05.556446 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 10 12:19:05.556700 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:05.556689 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wdbdh\"" Sep 10 12:19:05.764466 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:05.764390 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ljkj8"] Sep 10 12:19:05.769484 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:19:05.768370 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67095b3f_a8e2_494b_9f83_b4eb0b5634a2.slice/crio-94d8675c73ef9500afe21f3cb62618a3f102e029749613d17a2788920d3befdc WatchSource:0}: Error finding container 94d8675c73ef9500afe21f3cb62618a3f102e029749613d17a2788920d3befdc: Status 404 returned error can't find the container with id 94d8675c73ef9500afe21f3cb62618a3f102e029749613d17a2788920d3befdc Sep 10 12:19:05.892236 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:05.892177 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ljkj8" event={"ID":"67095b3f-a8e2-494b-9f83-b4eb0b5634a2","Type":"ContainerStarted","Data":"94d8675c73ef9500afe21f3cb62618a3f102e029749613d17a2788920d3befdc"} Sep 10 12:19:05.894503 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:05.894481 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t5mdr" event={"ID":"5b1d6ed3-daa3-4642-84d3-dd8cfe53e110","Type":"ContainerStarted","Data":"0153c76060d70e89be54c5c2935f11acdb15565aa22fa5462234375123725885"} Sep 10 12:19:08.906836 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:08.906744 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2f4lc" event={"ID":"e09b8c21-6ece-4589-8fa2-dfa2c0c0afe4","Type":"ContainerStarted","Data":"7bc218ec5498d62b0cbbb5871a7cfce4e0a161e9c577657836cc54c0c3e77330"} Sep 10 12:19:08.909418 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:08.909384 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t5mdr" event={"ID":"5b1d6ed3-daa3-4642-84d3-dd8cfe53e110","Type":"ContainerStarted","Data":"c18c552073c3ebd8dd6ba3537181183dde53a1916a32b9263f3be0b71e718a0e"} Sep 10 12:19:08.911498 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:08.911470 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sjwmh" event={"ID":"42e548ce-35b0-4117-952f-c0b6bec98b14","Type":"ContainerStarted","Data":"3c5225ce9739f7d2f8d4d1745c2a6e78fb26ccfd45a382a72ca482f44c4b3051"} Sep 10 12:19:08.911632 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:08.911519 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sjwmh" event={"ID":"42e548ce-35b0-4117-952f-c0b6bec98b14","Type":"ContainerStarted","Data":"3ca5e042b7e7c614702c99f9c92791a1532c510e634a60aede8d2d2bf99ce293"} Sep 10 12:19:08.911632 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:08.911615 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-sjwmh" Sep 10 12:19:08.927955 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:08.927848 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-2f4lc" podStartSLOduration=2.125716892 podStartE2EDuration="5.927830297s" totalImagesPullingTime="3.802113405s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:19:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:19:08.927625716 +0000 UTC m=+68.934942100" watchObservedRunningTime="2026-09-10 12:19:08.927830297 +0000 UTC m=+68.935146686" Sep 10 12:19:08.948257 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:08.948202 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-sjwmh" podStartSLOduration=2.110369627 podStartE2EDuration="5.948184742s" totalImagesPullingTime="3.837815115s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:19:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:19:08.947365623 +0000 UTC m=+68.954682094" watchObservedRunningTime="2026-09-10 12:19:08.948184742 +0000 UTC m=+68.955501127" Sep 10 12:19:08.968271 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:08.968220 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-t5mdr" podStartSLOduration=2.372011472 podStartE2EDuration="5.968204134s" totalImagesPullingTime="3.596192662s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:19:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:19:08.967225002 +0000 UTC m=+68.974541588" watchObservedRunningTime="2026-09-10 12:19:08.968204134 +0000 UTC m=+68.975520530" Sep 10 12:19:10.845702 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:10.845674 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tfcmz" Sep 10 12:19:11.156098 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.156021 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nt7ht"] Sep 10 12:19:11.158298 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.158278 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nt7ht" Sep 10 12:19:11.161368 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.161346 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Sep 10 12:19:11.161487 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.161391 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-tcssp\"" Sep 10 12:19:11.168524 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.168501 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nt7ht"] Sep 10 12:19:11.244536 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.244486 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5994acc2-09f9-40fe-835f-07f49c9fec9d-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-nt7ht\" (UID: \"5994acc2-09f9-40fe-835f-07f49c9fec9d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nt7ht" Sep 10 12:19:11.345409 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.345374 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5994acc2-09f9-40fe-835f-07f49c9fec9d-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-nt7ht\" (UID: \"5994acc2-09f9-40fe-835f-07f49c9fec9d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nt7ht" Sep 10 12:19:11.345582 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:19:11.345549 2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Sep 10 12:19:11.345648 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:19:11.345631 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5994acc2-09f9-40fe-835f-07f49c9fec9d-tls-certificates podName:5994acc2-09f9-40fe-835f-07f49c9fec9d nodeName:}" failed. No retries permitted until 2026-09-10 12:19:11.845608171 +0000 UTC m=+71.852924533 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/5994acc2-09f9-40fe-835f-07f49c9fec9d-tls-certificates") pod "prometheus-operator-admission-webhook-78f599db79-nt7ht" (UID: "5994acc2-09f9-40fe-835f-07f49c9fec9d") : secret "prometheus-operator-admission-webhook-tls" not found Sep 10 12:19:11.619253 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.619220 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b565596bf-z8wcz"] Sep 10 12:19:11.621270 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.621240 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b565596bf-z8wcz" Sep 10 12:19:11.624497 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.624447 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-cl8sn\"" Sep 10 12:19:11.624838 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.624810 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 10 12:19:11.625485 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.625199 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 10 12:19:11.625485 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.625265 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 10 12:19:11.625485 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.625206 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 10 12:19:11.625485 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.625484 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 10 12:19:11.625889 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.625515 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 10 12:19:11.625978 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.625963 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 10 12:19:11.630177 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.630160 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 10 12:19:11.638605 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.638585 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b565596bf-z8wcz"] Sep 10 12:19:11.749136 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.749095 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/afdcb83e-8517-4833-af04-22feff9dc221-service-ca\") pod \"console-6b565596bf-z8wcz\" (UID: \"afdcb83e-8517-4833-af04-22feff9dc221\") " pod="openshift-console/console-6b565596bf-z8wcz" Sep 10 12:19:11.749324 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.749152 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/afdcb83e-8517-4833-af04-22feff9dc221-oauth-serving-cert\") pod \"console-6b565596bf-z8wcz\" (UID: \"afdcb83e-8517-4833-af04-22feff9dc221\") " pod="openshift-console/console-6b565596bf-z8wcz" Sep 10 12:19:11.749324 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.749263 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/afdcb83e-8517-4833-af04-22feff9dc221-console-serving-cert\") pod \"console-6b565596bf-z8wcz\" (UID: \"afdcb83e-8517-4833-af04-22feff9dc221\") " pod="openshift-console/console-6b565596bf-z8wcz" Sep 10 12:19:11.749324 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.749294 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/afdcb83e-8517-4833-af04-22feff9dc221-console-oauth-config\") pod \"console-6b565596bf-z8wcz\" (UID: \"afdcb83e-8517-4833-af04-22feff9dc221\") " pod="openshift-console/console-6b565596bf-z8wcz" Sep 10 12:19:11.749487 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.749342 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8jnx\" (UniqueName: \"kubernetes.io/projected/afdcb83e-8517-4833-af04-22feff9dc221-kube-api-access-d8jnx\") pod \"console-6b565596bf-z8wcz\" (UID: \"afdcb83e-8517-4833-af04-22feff9dc221\") " pod="openshift-console/console-6b565596bf-z8wcz" Sep 10 12:19:11.749487 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.749377 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/afdcb83e-8517-4833-af04-22feff9dc221-console-config\") pod \"console-6b565596bf-z8wcz\" (UID: \"afdcb83e-8517-4833-af04-22feff9dc221\") " pod="openshift-console/console-6b565596bf-z8wcz" Sep 10 12:19:11.749487 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.749407 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afdcb83e-8517-4833-af04-22feff9dc221-trusted-ca-bundle\") pod \"console-6b565596bf-z8wcz\" (UID: \"afdcb83e-8517-4833-af04-22feff9dc221\") " pod="openshift-console/console-6b565596bf-z8wcz" Sep 10 12:19:11.849993 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.849958 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afdcb83e-8517-4833-af04-22feff9dc221-trusted-ca-bundle\") pod \"console-6b565596bf-z8wcz\" (UID: \"afdcb83e-8517-4833-af04-22feff9dc221\") " pod="openshift-console/console-6b565596bf-z8wcz" Sep 10 12:19:11.850390 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.850014 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5994acc2-09f9-40fe-835f-07f49c9fec9d-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-nt7ht\" (UID: \"5994acc2-09f9-40fe-835f-07f49c9fec9d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nt7ht" Sep 10 12:19:11.850390 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.850043 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/afdcb83e-8517-4833-af04-22feff9dc221-service-ca\") pod \"console-6b565596bf-z8wcz\" (UID: \"afdcb83e-8517-4833-af04-22feff9dc221\") " pod="openshift-console/console-6b565596bf-z8wcz" Sep 10 12:19:11.850390 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.850069 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/afdcb83e-8517-4833-af04-22feff9dc221-oauth-serving-cert\") pod \"console-6b565596bf-z8wcz\" (UID: \"afdcb83e-8517-4833-af04-22feff9dc221\") " pod="openshift-console/console-6b565596bf-z8wcz" Sep 10 12:19:11.850390 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.850120 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/afdcb83e-8517-4833-af04-22feff9dc221-console-serving-cert\") pod \"console-6b565596bf-z8wcz\" (UID: \"afdcb83e-8517-4833-af04-22feff9dc221\") " pod="openshift-console/console-6b565596bf-z8wcz" Sep 10 12:19:11.850390 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:19:11.850134 2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Sep 10 12:19:11.850390 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:19:11.850202 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5994acc2-09f9-40fe-835f-07f49c9fec9d-tls-certificates podName:5994acc2-09f9-40fe-835f-07f49c9fec9d nodeName:}" failed. No retries permitted until 2026-09-10 12:19:12.850183962 +0000 UTC m=+72.857500327 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/5994acc2-09f9-40fe-835f-07f49c9fec9d-tls-certificates") pod "prometheus-operator-admission-webhook-78f599db79-nt7ht" (UID: "5994acc2-09f9-40fe-835f-07f49c9fec9d") : secret "prometheus-operator-admission-webhook-tls" not found Sep 10 12:19:11.850710 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.850510 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/afdcb83e-8517-4833-af04-22feff9dc221-console-oauth-config\") pod \"console-6b565596bf-z8wcz\" (UID: \"afdcb83e-8517-4833-af04-22feff9dc221\") " pod="openshift-console/console-6b565596bf-z8wcz" Sep 10 12:19:11.850710 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.850565 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8jnx\" (UniqueName: \"kubernetes.io/projected/afdcb83e-8517-4833-af04-22feff9dc221-kube-api-access-d8jnx\") pod \"console-6b565596bf-z8wcz\" (UID: \"afdcb83e-8517-4833-af04-22feff9dc221\") " pod="openshift-console/console-6b565596bf-z8wcz" Sep 10 12:19:11.850710 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.850596 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/afdcb83e-8517-4833-af04-22feff9dc221-console-config\") pod \"console-6b565596bf-z8wcz\" (UID: \"afdcb83e-8517-4833-af04-22feff9dc221\") " pod="openshift-console/console-6b565596bf-z8wcz" Sep 10 12:19:11.851259 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.850934 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/afdcb83e-8517-4833-af04-22feff9dc221-service-ca\") pod \"console-6b565596bf-z8wcz\" (UID: \"afdcb83e-8517-4833-af04-22feff9dc221\") " pod="openshift-console/console-6b565596bf-z8wcz" Sep 10 12:19:11.851259 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.851075 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/afdcb83e-8517-4833-af04-22feff9dc221-oauth-serving-cert\") pod \"console-6b565596bf-z8wcz\" (UID: \"afdcb83e-8517-4833-af04-22feff9dc221\") " pod="openshift-console/console-6b565596bf-z8wcz" Sep 10 12:19:11.851400 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.851288 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/afdcb83e-8517-4833-af04-22feff9dc221-console-config\") pod \"console-6b565596bf-z8wcz\" (UID: \"afdcb83e-8517-4833-af04-22feff9dc221\") " pod="openshift-console/console-6b565596bf-z8wcz" Sep 10 12:19:11.851454 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.851411 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afdcb83e-8517-4833-af04-22feff9dc221-trusted-ca-bundle\") pod \"console-6b565596bf-z8wcz\" (UID: \"afdcb83e-8517-4833-af04-22feff9dc221\") " pod="openshift-console/console-6b565596bf-z8wcz" Sep 10 12:19:11.852890 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.852846 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/afdcb83e-8517-4833-af04-22feff9dc221-console-serving-cert\") pod \"console-6b565596bf-z8wcz\" (UID: \"afdcb83e-8517-4833-af04-22feff9dc221\") " pod="openshift-console/console-6b565596bf-z8wcz" Sep 10 12:19:11.853636 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.853615 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/afdcb83e-8517-4833-af04-22feff9dc221-console-oauth-config\") pod \"console-6b565596bf-z8wcz\" (UID: \"afdcb83e-8517-4833-af04-22feff9dc221\") " pod="openshift-console/console-6b565596bf-z8wcz" Sep 10 12:19:11.863163 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.863135 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8jnx\" (UniqueName: \"kubernetes.io/projected/afdcb83e-8517-4833-af04-22feff9dc221-kube-api-access-d8jnx\") pod \"console-6b565596bf-z8wcz\" (UID: \"afdcb83e-8517-4833-af04-22feff9dc221\") " pod="openshift-console/console-6b565596bf-z8wcz" Sep 10 12:19:11.933075 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:11.933046 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b565596bf-z8wcz" Sep 10 12:19:12.211434 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:12.211403 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b565596bf-z8wcz"] Sep 10 12:19:12.214378 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:19:12.214349 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafdcb83e_8517_4833_af04_22feff9dc221.slice/crio-e349ade2aaea18772f3681e2acc4eecb19f0fb2cfba498cefd24cf5bd23dba2c WatchSource:0}: Error finding container e349ade2aaea18772f3681e2acc4eecb19f0fb2cfba498cefd24cf5bd23dba2c: Status 404 returned error can't find the container with id e349ade2aaea18772f3681e2acc4eecb19f0fb2cfba498cefd24cf5bd23dba2c Sep 10 12:19:12.857655 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:12.857625 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5994acc2-09f9-40fe-835f-07f49c9fec9d-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-nt7ht\" (UID: \"5994acc2-09f9-40fe-835f-07f49c9fec9d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nt7ht" Sep 10 12:19:12.859958 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:12.859931 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5994acc2-09f9-40fe-835f-07f49c9fec9d-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-nt7ht\" (UID: \"5994acc2-09f9-40fe-835f-07f49c9fec9d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nt7ht" Sep 10 12:19:12.924673 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:12.924641 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b565596bf-z8wcz" event={"ID":"afdcb83e-8517-4833-af04-22feff9dc221","Type":"ContainerStarted","Data":"e349ade2aaea18772f3681e2acc4eecb19f0fb2cfba498cefd24cf5bd23dba2c"} Sep 10 12:19:12.926082 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:12.926060 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ljkj8" event={"ID":"67095b3f-a8e2-494b-9f83-b4eb0b5634a2","Type":"ContainerStarted","Data":"35ee9a10f2a424940230f46fc78c7d4a7182a76946ef218a2c28f7e8b61a2086"} Sep 10 12:19:12.942952 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:12.942912 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-ljkj8" podStartSLOduration=66.880568634 podStartE2EDuration="1m12.942895422s" totalImagesPullingTime="6.062326788s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:18:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:19:12.94268152 +0000 UTC m=+72.949997903" watchObservedRunningTime="2026-09-10 12:19:12.942895422 +0000 UTC m=+72.950211796" Sep 10 12:19:12.969135 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:12.969107 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nt7ht" Sep 10 12:19:13.236413 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:13.236372 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nt7ht"] Sep 10 12:19:13.240411 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:19:13.240385 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5994acc2_09f9_40fe_835f_07f49c9fec9d.slice/crio-f18f332e8e7f13ec1d98aca41d33ae7c3d4376b8774f359c1e35e1b1ac1c6215 WatchSource:0}: Error finding container f18f332e8e7f13ec1d98aca41d33ae7c3d4376b8774f359c1e35e1b1ac1c6215: Status 404 returned error can't find the container with id f18f332e8e7f13ec1d98aca41d33ae7c3d4376b8774f359c1e35e1b1ac1c6215 Sep 10 12:19:13.931519 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:13.931482 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nt7ht" event={"ID":"5994acc2-09f9-40fe-835f-07f49c9fec9d","Type":"ContainerStarted","Data":"f18f332e8e7f13ec1d98aca41d33ae7c3d4376b8774f359c1e35e1b1ac1c6215"} Sep 10 12:19:16.945563 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:16.945527 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nt7ht" event={"ID":"5994acc2-09f9-40fe-835f-07f49c9fec9d","Type":"ContainerStarted","Data":"72bc2dbbad21c58511d3a422a26792997ad18a1b9c0045a1b311b255b40ffa04"} Sep 10 12:19:16.946035 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:16.945717 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nt7ht" Sep 10 12:19:16.947255 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:16.947230 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b565596bf-z8wcz" event={"ID":"afdcb83e-8517-4833-af04-22feff9dc221","Type":"ContainerStarted","Data":"3741c1eb9bd3cc9f4f42301d40d04e04aa31f353f3c62f50e3b48e2808480286"} Sep 10 12:19:16.950740 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:16.950715 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nt7ht" Sep 10 12:19:16.965340 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:16.965303 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nt7ht" podStartSLOduration=3.321540086 podStartE2EDuration="5.965290208s" totalImagesPullingTime="2.643750122s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:19:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:19:16.964664612 +0000 UTC m=+76.971980995" watchObservedRunningTime="2026-09-10 12:19:16.965290208 +0000 UTC m=+76.972606593" Sep 10 12:19:17.221148 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:17.221045 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b565596bf-z8wcz" podStartSLOduration=2.545249902 podStartE2EDuration="6.221027078s" totalImagesPullingTime="3.675777176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:19:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:19:17.048938037 +0000 UTC m=+77.056254420" watchObservedRunningTime="2026-09-10 12:19:17.221027078 +0000 UTC m=+77.228343464" Sep 10 12:19:17.221756 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:17.221734 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-npsn2"] Sep 10 12:19:17.223562 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:17.223542 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-npsn2" Sep 10 12:19:17.226775 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:17.226744 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 10 12:19:17.226915 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:17.226781 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 10 12:19:17.226915 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:17.226786 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 10 12:19:17.226915 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:17.226840 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 10 12:19:17.227040 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:17.226999 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 10 12:19:17.227098 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:17.227082 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-82hdn\"" Sep 10 12:19:17.236221 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:17.236203 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-npsn2"] Sep 10 12:19:17.291494 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:17.291466 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k98qs\" (UniqueName: \"kubernetes.io/projected/7f6e361b-caa6-49bb-876e-a21a57badd11-kube-api-access-k98qs\") pod \"prometheus-operator-6f4946db56-npsn2\" (UID: \"7f6e361b-caa6-49bb-876e-a21a57badd11\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-npsn2" Sep 10 12:19:17.291632 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:17.291499 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f6e361b-caa6-49bb-876e-a21a57badd11-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-npsn2\" (UID: \"7f6e361b-caa6-49bb-876e-a21a57badd11\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-npsn2" Sep 10 12:19:17.291632 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:17.291589 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f6e361b-caa6-49bb-876e-a21a57badd11-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-npsn2\" (UID: \"7f6e361b-caa6-49bb-876e-a21a57badd11\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-npsn2" Sep 10 12:19:17.291632 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:17.291618 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f6e361b-caa6-49bb-876e-a21a57badd11-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-npsn2\" (UID: \"7f6e361b-caa6-49bb-876e-a21a57badd11\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-npsn2" Sep 10 12:19:17.392663 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:17.392634 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f6e361b-caa6-49bb-876e-a21a57badd11-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-npsn2\" (UID: \"7f6e361b-caa6-49bb-876e-a21a57badd11\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-npsn2" Sep 10 12:19:17.392746 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:17.392667 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f6e361b-caa6-49bb-876e-a21a57badd11-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-npsn2\" (UID: \"7f6e361b-caa6-49bb-876e-a21a57badd11\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-npsn2" Sep 10 12:19:17.392746 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:17.392701 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k98qs\" (UniqueName: \"kubernetes.io/projected/7f6e361b-caa6-49bb-876e-a21a57badd11-kube-api-access-k98qs\") pod \"prometheus-operator-6f4946db56-npsn2\" (UID: \"7f6e361b-caa6-49bb-876e-a21a57badd11\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-npsn2" Sep 10 12:19:17.392842 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:17.392820 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f6e361b-caa6-49bb-876e-a21a57badd11-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-npsn2\" (UID: \"7f6e361b-caa6-49bb-876e-a21a57badd11\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-npsn2" Sep 10 12:19:17.393375 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:17.393356 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f6e361b-caa6-49bb-876e-a21a57badd11-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-npsn2\" (UID: \"7f6e361b-caa6-49bb-876e-a21a57badd11\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-npsn2" Sep 10 12:19:17.395071 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:17.395050 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f6e361b-caa6-49bb-876e-a21a57badd11-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-npsn2\" (UID: \"7f6e361b-caa6-49bb-876e-a21a57badd11\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-npsn2" Sep 10 12:19:17.395135 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:17.395121 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f6e361b-caa6-49bb-876e-a21a57badd11-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-npsn2\" (UID: \"7f6e361b-caa6-49bb-876e-a21a57badd11\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-npsn2" Sep 10 12:19:17.401798 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:17.401773 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k98qs\" (UniqueName: \"kubernetes.io/projected/7f6e361b-caa6-49bb-876e-a21a57badd11-kube-api-access-k98qs\") pod \"prometheus-operator-6f4946db56-npsn2\" (UID: \"7f6e361b-caa6-49bb-876e-a21a57badd11\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-npsn2" Sep 10 12:19:17.532100 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:17.532017 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-npsn2" Sep 10 12:19:17.800347 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:17.800272 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-npsn2"] Sep 10 12:19:17.803691 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:19:17.803662 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f6e361b_caa6_49bb_876e_a21a57badd11.slice/crio-71392c29995061d839b3cf7fc7317201ac77f461382e73fd807ae73d2f1b92a3 WatchSource:0}: Error finding container 71392c29995061d839b3cf7fc7317201ac77f461382e73fd807ae73d2f1b92a3: Status 404 returned error can't find the container with id 71392c29995061d839b3cf7fc7317201ac77f461382e73fd807ae73d2f1b92a3 Sep 10 12:19:17.951689 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:17.951653 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-npsn2" event={"ID":"7f6e361b-caa6-49bb-876e-a21a57badd11","Type":"ContainerStarted","Data":"71392c29995061d839b3cf7fc7317201ac77f461382e73fd807ae73d2f1b92a3"} Sep 10 12:19:18.917562 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:18.917530 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-sjwmh" Sep 10 12:19:19.960996 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:19.960965 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-npsn2" event={"ID":"7f6e361b-caa6-49bb-876e-a21a57badd11","Type":"ContainerStarted","Data":"c839219c71eaac3dbe2486a907024f593930d1819f1479a68872593f4e5b04fc"} Sep 10 12:19:19.961364 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:19.961020 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-npsn2" event={"ID":"7f6e361b-caa6-49bb-876e-a21a57badd11","Type":"ContainerStarted","Data":"fb67e01f462f062971edf3f090072d3f08c47979affe0aa464e1fb06f7448790"} Sep 10 12:19:19.982812 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:19.982756 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-npsn2" podStartSLOduration=1.5132147649999999 podStartE2EDuration="2.982728151s" totalImagesPullingTime="1.469513386s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:19:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:19:19.982329383 +0000 UTC m=+79.989645765" watchObservedRunningTime="2026-09-10 12:19:19.982728151 +0000 UTC m=+79.990044537" Sep 10 12:19:21.615451 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.615407 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-wxwzn"] Sep 10 12:19:21.618460 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.618443 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-wxwzn" Sep 10 12:19:21.621608 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.621589 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 10 12:19:21.621712 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.621592 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-bzs8q\"" Sep 10 12:19:21.623380 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.623358 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 10 12:19:21.637047 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.637023 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-wxwzn"] Sep 10 12:19:21.642808 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.642781 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-krrfz"] Sep 10 12:19:21.646148 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.646129 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-krrfz" Sep 10 12:19:21.649175 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.649160 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-r8bjh\"" Sep 10 12:19:21.650046 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.650028 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Sep 10 12:19:21.650140 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.650045 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Sep 10 12:19:21.650140 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.650058 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Sep 10 12:19:21.657412 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.657395 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-b6452"] Sep 10 12:19:21.660272 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.660258 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b6452" Sep 10 12:19:21.663286 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.663262 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-krrfz"] Sep 10 12:19:21.663388 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.663373 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 10 12:19:21.663431 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.663395 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 10 12:19:21.663662 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.663645 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 10 12:19:21.664878 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.664845 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xpp5p\"" Sep 10 12:19:21.726470 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.726441 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/807c357a-a718-4e5e-8af5-29f075c8c276-node-exporter-textfile\") pod \"node-exporter-b6452\" (UID: \"807c357a-a718-4e5e-8af5-29f075c8c276\") " pod="openshift-monitoring/node-exporter-b6452" Sep 10 12:19:21.726625 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.726482 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/807c357a-a718-4e5e-8af5-29f075c8c276-root\") pod \"node-exporter-b6452\" (UID: \"807c357a-a718-4e5e-8af5-29f075c8c276\") " pod="openshift-monitoring/node-exporter-b6452" Sep 10 12:19:21.726625 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.726510 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/226b1299-cb0c-4568-8ae4-b3d955914d15-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-krrfz\" (UID: \"226b1299-cb0c-4568-8ae4-b3d955914d15\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-krrfz" Sep 10 12:19:21.726625 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.726526 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/807c357a-a718-4e5e-8af5-29f075c8c276-node-exporter-wtmp\") pod \"node-exporter-b6452\" (UID: \"807c357a-a718-4e5e-8af5-29f075c8c276\") " pod="openshift-monitoring/node-exporter-b6452" Sep 10 12:19:21.726625 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.726551 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8m2xp\" (UniqueName: \"kubernetes.io/projected/226b1299-cb0c-4568-8ae4-b3d955914d15-kube-api-access-8m2xp\") pod \"kube-state-metrics-74cb57df7-krrfz\" (UID: \"226b1299-cb0c-4568-8ae4-b3d955914d15\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-krrfz" Sep 10 12:19:21.726625 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.726568 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/807c357a-a718-4e5e-8af5-29f075c8c276-node-exporter-accelerators-collector-config\") pod \"node-exporter-b6452\" (UID: \"807c357a-a718-4e5e-8af5-29f075c8c276\") " pod="openshift-monitoring/node-exporter-b6452" Sep 10 12:19:21.726625 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.726586 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/807c357a-a718-4e5e-8af5-29f075c8c276-sys\") pod \"node-exporter-b6452\" (UID: \"807c357a-a718-4e5e-8af5-29f075c8c276\") " pod="openshift-monitoring/node-exporter-b6452" Sep 10 12:19:21.726625 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.726600 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/807c357a-a718-4e5e-8af5-29f075c8c276-metrics-client-ca\") pod \"node-exporter-b6452\" (UID: \"807c357a-a718-4e5e-8af5-29f075c8c276\") " pod="openshift-monitoring/node-exporter-b6452" Sep 10 12:19:21.726625 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.726617 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5tqk\" (UniqueName: \"kubernetes.io/projected/807c357a-a718-4e5e-8af5-29f075c8c276-kube-api-access-r5tqk\") pod \"node-exporter-b6452\" (UID: \"807c357a-a718-4e5e-8af5-29f075c8c276\") " pod="openshift-monitoring/node-exporter-b6452" Sep 10 12:19:21.726896 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.726669 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/226b1299-cb0c-4568-8ae4-b3d955914d15-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-krrfz\" (UID: \"226b1299-cb0c-4568-8ae4-b3d955914d15\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-krrfz" Sep 10 12:19:21.726896 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.726687 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/464a50e2-1786-451d-b7c5-567d20b5972f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-wxwzn\" (UID: \"464a50e2-1786-451d-b7c5-567d20b5972f\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-wxwzn" Sep 10 12:19:21.726896 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.726703 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/464a50e2-1786-451d-b7c5-567d20b5972f-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-wxwzn\" (UID: \"464a50e2-1786-451d-b7c5-567d20b5972f\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-wxwzn" Sep 10 12:19:21.726896 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.726730 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/807c357a-a718-4e5e-8af5-29f075c8c276-node-exporter-tls\") pod \"node-exporter-b6452\" (UID: \"807c357a-a718-4e5e-8af5-29f075c8c276\") " pod="openshift-monitoring/node-exporter-b6452" Sep 10 12:19:21.726896 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.726745 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/226b1299-cb0c-4568-8ae4-b3d955914d15-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-krrfz\" (UID: \"226b1299-cb0c-4568-8ae4-b3d955914d15\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-krrfz" Sep 10 12:19:21.726896 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.726769 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/226b1299-cb0c-4568-8ae4-b3d955914d15-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-krrfz\" (UID: \"226b1299-cb0c-4568-8ae4-b3d955914d15\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-krrfz" Sep 10 12:19:21.726896 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.726786 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/807c357a-a718-4e5e-8af5-29f075c8c276-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b6452\" (UID: \"807c357a-a718-4e5e-8af5-29f075c8c276\") " pod="openshift-monitoring/node-exporter-b6452" Sep 10 12:19:21.726896 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.726804 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/464a50e2-1786-451d-b7c5-567d20b5972f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-wxwzn\" (UID: \"464a50e2-1786-451d-b7c5-567d20b5972f\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-wxwzn" Sep 10 12:19:21.726896 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.726820 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xhx7\" (UniqueName: \"kubernetes.io/projected/464a50e2-1786-451d-b7c5-567d20b5972f-kube-api-access-4xhx7\") pod \"openshift-state-metrics-657fd6c54-wxwzn\" (UID: \"464a50e2-1786-451d-b7c5-567d20b5972f\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-wxwzn" Sep 10 12:19:21.726896 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.726842 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/226b1299-cb0c-4568-8ae4-b3d955914d15-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-krrfz\" (UID: \"226b1299-cb0c-4568-8ae4-b3d955914d15\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-krrfz" Sep 10 12:19:21.828245 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.828201 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/464a50e2-1786-451d-b7c5-567d20b5972f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-wxwzn\" (UID: \"464a50e2-1786-451d-b7c5-567d20b5972f\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-wxwzn" Sep 10 12:19:21.828245 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.828246 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xhx7\" (UniqueName: \"kubernetes.io/projected/464a50e2-1786-451d-b7c5-567d20b5972f-kube-api-access-4xhx7\") pod \"openshift-state-metrics-657fd6c54-wxwzn\" (UID: \"464a50e2-1786-451d-b7c5-567d20b5972f\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-wxwzn" Sep 10 12:19:21.828458 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.828273 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/226b1299-cb0c-4568-8ae4-b3d955914d15-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-krrfz\" (UID: \"226b1299-cb0c-4568-8ae4-b3d955914d15\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-krrfz" Sep 10 12:19:21.828458 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.828316 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/807c357a-a718-4e5e-8af5-29f075c8c276-node-exporter-textfile\") pod \"node-exporter-b6452\" (UID: \"807c357a-a718-4e5e-8af5-29f075c8c276\") " pod="openshift-monitoring/node-exporter-b6452" Sep 10 12:19:21.828458 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.828360 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/807c357a-a718-4e5e-8af5-29f075c8c276-root\") pod \"node-exporter-b6452\" (UID: \"807c357a-a718-4e5e-8af5-29f075c8c276\") " pod="openshift-monitoring/node-exporter-b6452" Sep 10 12:19:21.828458 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.828398 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/226b1299-cb0c-4568-8ae4-b3d955914d15-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-krrfz\" (UID: \"226b1299-cb0c-4568-8ae4-b3d955914d15\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-krrfz" Sep 10 12:19:21.828666 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.828481 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/807c357a-a718-4e5e-8af5-29f075c8c276-node-exporter-wtmp\") pod \"node-exporter-b6452\" (UID: \"807c357a-a718-4e5e-8af5-29f075c8c276\") " pod="openshift-monitoring/node-exporter-b6452" Sep 10 12:19:21.828666 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.828503 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/807c357a-a718-4e5e-8af5-29f075c8c276-root\") pod \"node-exporter-b6452\" (UID: \"807c357a-a718-4e5e-8af5-29f075c8c276\") " pod="openshift-monitoring/node-exporter-b6452" Sep 10 12:19:21.828666 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.828513 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8m2xp\" (UniqueName: \"kubernetes.io/projected/226b1299-cb0c-4568-8ae4-b3d955914d15-kube-api-access-8m2xp\") pod \"kube-state-metrics-74cb57df7-krrfz\" (UID: \"226b1299-cb0c-4568-8ae4-b3d955914d15\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-krrfz" Sep 10 12:19:21.828666 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.828569 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/807c357a-a718-4e5e-8af5-29f075c8c276-node-exporter-accelerators-collector-config\") pod \"node-exporter-b6452\" (UID: \"807c357a-a718-4e5e-8af5-29f075c8c276\") " pod="openshift-monitoring/node-exporter-b6452" Sep 10 12:19:21.828666 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.828604 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/807c357a-a718-4e5e-8af5-29f075c8c276-sys\") pod \"node-exporter-b6452\" (UID: \"807c357a-a718-4e5e-8af5-29f075c8c276\") " pod="openshift-monitoring/node-exporter-b6452" Sep 10 12:19:21.828666 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.828646 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/807c357a-a718-4e5e-8af5-29f075c8c276-metrics-client-ca\") pod \"node-exporter-b6452\" (UID: \"807c357a-a718-4e5e-8af5-29f075c8c276\") " pod="openshift-monitoring/node-exporter-b6452" Sep 10 12:19:21.828989 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.828679 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5tqk\" (UniqueName: \"kubernetes.io/projected/807c357a-a718-4e5e-8af5-29f075c8c276-kube-api-access-r5tqk\") pod \"node-exporter-b6452\" (UID: \"807c357a-a718-4e5e-8af5-29f075c8c276\") " pod="openshift-monitoring/node-exporter-b6452" Sep 10 12:19:21.828989 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.828724 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/226b1299-cb0c-4568-8ae4-b3d955914d15-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-krrfz\" (UID: \"226b1299-cb0c-4568-8ae4-b3d955914d15\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-krrfz" Sep 10 12:19:21.828989 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.828730 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/807c357a-a718-4e5e-8af5-29f075c8c276-node-exporter-wtmp\") pod \"node-exporter-b6452\" (UID: \"807c357a-a718-4e5e-8af5-29f075c8c276\") " pod="openshift-monitoring/node-exporter-b6452" Sep 10 12:19:21.828989 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.828761 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/464a50e2-1786-451d-b7c5-567d20b5972f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-wxwzn\" (UID: \"464a50e2-1786-451d-b7c5-567d20b5972f\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-wxwzn" Sep 10 12:19:21.828989 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.828788 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/807c357a-a718-4e5e-8af5-29f075c8c276-sys\") pod \"node-exporter-b6452\" (UID: \"807c357a-a718-4e5e-8af5-29f075c8c276\") " pod="openshift-monitoring/node-exporter-b6452" Sep 10 12:19:21.828989 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.828787 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/464a50e2-1786-451d-b7c5-567d20b5972f-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-wxwzn\" (UID: \"464a50e2-1786-451d-b7c5-567d20b5972f\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-wxwzn" Sep 10 12:19:21.828989 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.828835 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/807c357a-a718-4e5e-8af5-29f075c8c276-node-exporter-tls\") pod \"node-exporter-b6452\" (UID: \"807c357a-a718-4e5e-8af5-29f075c8c276\") " pod="openshift-monitoring/node-exporter-b6452" Sep 10 12:19:21.828989 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.828895 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/226b1299-cb0c-4568-8ae4-b3d955914d15-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-krrfz\" (UID: \"226b1299-cb0c-4568-8ae4-b3d955914d15\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-krrfz" Sep 10 12:19:21.828989 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.828922 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/226b1299-cb0c-4568-8ae4-b3d955914d15-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-krrfz\" (UID: \"226b1299-cb0c-4568-8ae4-b3d955914d15\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-krrfz" Sep 10 12:19:21.828989 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.828953 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/807c357a-a718-4e5e-8af5-29f075c8c276-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b6452\" (UID: \"807c357a-a718-4e5e-8af5-29f075c8c276\") " pod="openshift-monitoring/node-exporter-b6452" Sep 10 12:19:21.828989 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.828644 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/807c357a-a718-4e5e-8af5-29f075c8c276-node-exporter-textfile\") pod \"node-exporter-b6452\" (UID: \"807c357a-a718-4e5e-8af5-29f075c8c276\") " pod="openshift-monitoring/node-exporter-b6452" Sep 10 12:19:21.829523 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.829180 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/807c357a-a718-4e5e-8af5-29f075c8c276-node-exporter-accelerators-collector-config\") pod \"node-exporter-b6452\" (UID: \"807c357a-a718-4e5e-8af5-29f075c8c276\") " pod="openshift-monitoring/node-exporter-b6452" Sep 10 12:19:21.829523 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.828723 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/226b1299-cb0c-4568-8ae4-b3d955914d15-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-krrfz\" (UID: \"226b1299-cb0c-4568-8ae4-b3d955914d15\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-krrfz" Sep 10 12:19:21.829523 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.829248 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/807c357a-a718-4e5e-8af5-29f075c8c276-metrics-client-ca\") pod \"node-exporter-b6452\" (UID: \"807c357a-a718-4e5e-8af5-29f075c8c276\") " pod="openshift-monitoring/node-exporter-b6452" Sep 10 12:19:21.829523 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.829258 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/226b1299-cb0c-4568-8ae4-b3d955914d15-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-krrfz\" (UID: \"226b1299-cb0c-4568-8ae4-b3d955914d15\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-krrfz" Sep 10 12:19:21.829722 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.829563 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/464a50e2-1786-451d-b7c5-567d20b5972f-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-wxwzn\" (UID: \"464a50e2-1786-451d-b7c5-567d20b5972f\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-wxwzn" Sep 10 12:19:21.830124 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.830104 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/226b1299-cb0c-4568-8ae4-b3d955914d15-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-krrfz\" (UID: \"226b1299-cb0c-4568-8ae4-b3d955914d15\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-krrfz" Sep 10 12:19:21.831397 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.831376 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/464a50e2-1786-451d-b7c5-567d20b5972f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-wxwzn\" (UID: \"464a50e2-1786-451d-b7c5-567d20b5972f\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-wxwzn" Sep 10 12:19:21.831638 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.831614 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/226b1299-cb0c-4568-8ae4-b3d955914d15-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-krrfz\" (UID: \"226b1299-cb0c-4568-8ae4-b3d955914d15\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-krrfz" Sep 10 12:19:21.832071 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.832047 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/807c357a-a718-4e5e-8af5-29f075c8c276-node-exporter-tls\") pod \"node-exporter-b6452\" (UID: \"807c357a-a718-4e5e-8af5-29f075c8c276\") " pod="openshift-monitoring/node-exporter-b6452" Sep 10 12:19:21.832339 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.832078 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/807c357a-a718-4e5e-8af5-29f075c8c276-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b6452\" (UID: \"807c357a-a718-4e5e-8af5-29f075c8c276\") " pod="openshift-monitoring/node-exporter-b6452" Sep 10 12:19:21.832339 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.832111 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/226b1299-cb0c-4568-8ae4-b3d955914d15-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-krrfz\" (UID: \"226b1299-cb0c-4568-8ae4-b3d955914d15\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-krrfz" Sep 10 12:19:21.832339 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.832157 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/464a50e2-1786-451d-b7c5-567d20b5972f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-wxwzn\" (UID: \"464a50e2-1786-451d-b7c5-567d20b5972f\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-wxwzn" Sep 10 12:19:21.842841 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.842457 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5tqk\" (UniqueName: \"kubernetes.io/projected/807c357a-a718-4e5e-8af5-29f075c8c276-kube-api-access-r5tqk\") pod \"node-exporter-b6452\" (UID: \"807c357a-a718-4e5e-8af5-29f075c8c276\") " pod="openshift-monitoring/node-exporter-b6452" Sep 10 12:19:21.845504 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.845453 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8m2xp\" (UniqueName: \"kubernetes.io/projected/226b1299-cb0c-4568-8ae4-b3d955914d15-kube-api-access-8m2xp\") pod \"kube-state-metrics-74cb57df7-krrfz\" (UID: \"226b1299-cb0c-4568-8ae4-b3d955914d15\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-krrfz" Sep 10 12:19:21.863208 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.863178 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xhx7\" (UniqueName: \"kubernetes.io/projected/464a50e2-1786-451d-b7c5-567d20b5972f-kube-api-access-4xhx7\") pod \"openshift-state-metrics-657fd6c54-wxwzn\" (UID: \"464a50e2-1786-451d-b7c5-567d20b5972f\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-wxwzn" Sep 10 12:19:21.926919 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.926886 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-wxwzn" Sep 10 12:19:21.933845 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.933820 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b565596bf-z8wcz" Sep 10 12:19:21.933950 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.933883 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b565596bf-z8wcz" Sep 10 12:19:21.938592 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.938571 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b565596bf-z8wcz" Sep 10 12:19:21.954695 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.954674 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-krrfz" Sep 10 12:19:21.972423 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.972389 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b6452" Sep 10 12:19:21.977685 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:21.977491 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b565596bf-z8wcz" Sep 10 12:19:21.988000 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:19:21.987962 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod807c357a_a718_4e5e_8af5_29f075c8c276.slice/crio-b56a3c7e2c0b6a70fc283ddef77e7a03d45af1a96712624e97510619bf7334b6 WatchSource:0}: Error finding container b56a3c7e2c0b6a70fc283ddef77e7a03d45af1a96712624e97510619bf7334b6: Status 404 returned error can't find the container with id b56a3c7e2c0b6a70fc283ddef77e7a03d45af1a96712624e97510619bf7334b6 Sep 10 12:19:22.203564 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.203334 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-wxwzn"] Sep 10 12:19:22.205740 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:19:22.205713 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod464a50e2_1786_451d_b7c5_567d20b5972f.slice/crio-9931a79e363dec733a3a438ca2f0d3f0d7ffb9df85d69b923c2b508b45e52ccd WatchSource:0}: Error finding container 9931a79e363dec733a3a438ca2f0d3f0d7ffb9df85d69b923c2b508b45e52ccd: Status 404 returned error can't find the container with id 9931a79e363dec733a3a438ca2f0d3f0d7ffb9df85d69b923c2b508b45e52ccd Sep 10 12:19:22.246893 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.246850 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-krrfz"] Sep 10 12:19:22.250818 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:19:22.250793 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod226b1299_cb0c_4568_8ae4_b3d955914d15.slice/crio-2b41f136d297d8355f930b83f2ee5fa92adacdf834403c9b149b7cb1ad4de863 WatchSource:0}: Error finding container 2b41f136d297d8355f930b83f2ee5fa92adacdf834403c9b149b7cb1ad4de863: Status 404 returned error can't find the container with id 2b41f136d297d8355f930b83f2ee5fa92adacdf834403c9b149b7cb1ad4de863 Sep 10 12:19:22.756596 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.756560 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 12:19:22.760043 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.760027 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.765880 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.765838 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 10 12:19:22.766142 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.766121 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 10 12:19:22.766254 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.766140 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 10 12:19:22.766254 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.766154 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 10 12:19:22.766481 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.766419 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 10 12:19:22.766481 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.766442 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-7v47x\"" Sep 10 12:19:22.766622 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.766557 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 10 12:19:22.766676 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.766634 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 10 12:19:22.766957 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.766937 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 10 12:19:22.767041 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.767006 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 10 12:19:22.784441 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.784419 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 12:19:22.838484 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.838452 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f1f6afd1-7472-4c06-8651-f15801fab26d-config-volume\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.838664 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.838489 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f1f6afd1-7472-4c06-8651-f15801fab26d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.838664 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.838555 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f1f6afd1-7472-4c06-8651-f15801fab26d-config-out\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.838664 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.838581 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f1f6afd1-7472-4c06-8651-f15801fab26d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.838664 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.838603 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f1f6afd1-7472-4c06-8651-f15801fab26d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.838664 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.838649 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f1f6afd1-7472-4c06-8651-f15801fab26d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.838934 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.838675 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1f6afd1-7472-4c06-8651-f15801fab26d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.838934 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.838765 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f1f6afd1-7472-4c06-8651-f15801fab26d-web-config\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.838934 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.838811 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1f6afd1-7472-4c06-8651-f15801fab26d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.838934 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.838852 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f1f6afd1-7472-4c06-8651-f15801fab26d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.839138 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.838948 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f1f6afd1-7472-4c06-8651-f15801fab26d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.839138 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.838986 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82646\" (UniqueName: \"kubernetes.io/projected/f1f6afd1-7472-4c06-8651-f15801fab26d-kube-api-access-82646\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.839138 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.839011 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f1f6afd1-7472-4c06-8651-f15801fab26d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.940842 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.939724 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f1f6afd1-7472-4c06-8651-f15801fab26d-config-out\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.940842 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.939768 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f1f6afd1-7472-4c06-8651-f15801fab26d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.940842 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.939794 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f1f6afd1-7472-4c06-8651-f15801fab26d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.940842 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.939843 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f1f6afd1-7472-4c06-8651-f15801fab26d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.940842 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.939891 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1f6afd1-7472-4c06-8651-f15801fab26d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.940842 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.939918 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f1f6afd1-7472-4c06-8651-f15801fab26d-web-config\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.940842 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.939945 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1f6afd1-7472-4c06-8651-f15801fab26d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.940842 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.939978 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f1f6afd1-7472-4c06-8651-f15801fab26d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.940842 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.940032 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f1f6afd1-7472-4c06-8651-f15801fab26d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.940842 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.940067 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82646\" (UniqueName: \"kubernetes.io/projected/f1f6afd1-7472-4c06-8651-f15801fab26d-kube-api-access-82646\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.940842 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.940096 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f1f6afd1-7472-4c06-8651-f15801fab26d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.940842 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.940139 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f1f6afd1-7472-4c06-8651-f15801fab26d-config-volume\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.940842 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.940163 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f1f6afd1-7472-4c06-8651-f15801fab26d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.941681 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.941458 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f1f6afd1-7472-4c06-8651-f15801fab26d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.941681 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.941625 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1f6afd1-7472-4c06-8651-f15801fab26d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.943015 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.942990 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1f6afd1-7472-4c06-8651-f15801fab26d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.946526 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.946473 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f1f6afd1-7472-4c06-8651-f15801fab26d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.946849 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.946825 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f1f6afd1-7472-4c06-8651-f15801fab26d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.946945 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.946905 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f1f6afd1-7472-4c06-8651-f15801fab26d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.947115 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.947093 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f1f6afd1-7472-4c06-8651-f15801fab26d-config-out\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.948366 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.948324 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f1f6afd1-7472-4c06-8651-f15801fab26d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.949496 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.948760 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f1f6afd1-7472-4c06-8651-f15801fab26d-web-config\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.949496 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.949103 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f1f6afd1-7472-4c06-8651-f15801fab26d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.949496 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.949171 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f1f6afd1-7472-4c06-8651-f15801fab26d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.950371 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.950348 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f1f6afd1-7472-4c06-8651-f15801fab26d-config-volume\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.952217 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.952173 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82646\" (UniqueName: \"kubernetes.io/projected/f1f6afd1-7472-4c06-8651-f15801fab26d-kube-api-access-82646\") pod \"alertmanager-main-0\" (UID: \"f1f6afd1-7472-4c06-8651-f15801fab26d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:22.977749 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.977719 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-krrfz" event={"ID":"226b1299-cb0c-4568-8ae4-b3d955914d15","Type":"ContainerStarted","Data":"2b41f136d297d8355f930b83f2ee5fa92adacdf834403c9b149b7cb1ad4de863"} Sep 10 12:19:22.980772 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.980742 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-wxwzn" event={"ID":"464a50e2-1786-451d-b7c5-567d20b5972f","Type":"ContainerStarted","Data":"bb192a8f2e6dfc443b4188590b54a0da957ea7df778e9feae06318ab61f2621c"} Sep 10 12:19:22.980916 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.980803 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-wxwzn" event={"ID":"464a50e2-1786-451d-b7c5-567d20b5972f","Type":"ContainerStarted","Data":"4d558d9bfd364a535e1e52778cec6357f0775fc49532783ccb506f5ebfa4bec2"} Sep 10 12:19:22.980916 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.980831 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-wxwzn" event={"ID":"464a50e2-1786-451d-b7c5-567d20b5972f","Type":"ContainerStarted","Data":"9931a79e363dec733a3a438ca2f0d3f0d7ffb9df85d69b923c2b508b45e52ccd"} Sep 10 12:19:22.983756 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:22.983729 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b6452" event={"ID":"807c357a-a718-4e5e-8af5-29f075c8c276","Type":"ContainerStarted","Data":"b56a3c7e2c0b6a70fc283ddef77e7a03d45af1a96712624e97510619bf7334b6"} Sep 10 12:19:23.071037 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.070954 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 12:19:23.650077 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.650043 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-57c946f66b-qt62h"] Sep 10 12:19:23.653766 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.653745 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-57c946f66b-qt62h" Sep 10 12:19:23.656680 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.656638 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Sep 10 12:19:23.656829 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.656731 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-8dnbk\"" Sep 10 12:19:23.657253 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.657229 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Sep 10 12:19:23.657340 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.657271 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Sep 10 12:19:23.657579 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.657560 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Sep 10 12:19:23.661647 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.657883 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Sep 10 12:19:23.661647 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.658174 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-4uahapvsf8r1p\"" Sep 10 12:19:23.668463 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.668439 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-57c946f66b-qt62h"] Sep 10 12:19:23.749260 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.749223 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6675777c-d2f9-4754-8531-548d9088d313-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-57c946f66b-qt62h\" (UID: \"6675777c-d2f9-4754-8531-548d9088d313\") " pod="openshift-monitoring/thanos-querier-57c946f66b-qt62h" Sep 10 12:19:23.749444 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.749271 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6675777c-d2f9-4754-8531-548d9088d313-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-57c946f66b-qt62h\" (UID: \"6675777c-d2f9-4754-8531-548d9088d313\") " pod="openshift-monitoring/thanos-querier-57c946f66b-qt62h" Sep 10 12:19:23.749444 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.749337 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6675777c-d2f9-4754-8531-548d9088d313-secret-thanos-querier-tls\") pod \"thanos-querier-57c946f66b-qt62h\" (UID: \"6675777c-d2f9-4754-8531-548d9088d313\") " pod="openshift-monitoring/thanos-querier-57c946f66b-qt62h" Sep 10 12:19:23.749444 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.749417 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6675777c-d2f9-4754-8531-548d9088d313-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-57c946f66b-qt62h\" (UID: \"6675777c-d2f9-4754-8531-548d9088d313\") " pod="openshift-monitoring/thanos-querier-57c946f66b-qt62h" Sep 10 12:19:23.749615 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.749474 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6675777c-d2f9-4754-8531-548d9088d313-secret-grpc-tls\") pod \"thanos-querier-57c946f66b-qt62h\" (UID: \"6675777c-d2f9-4754-8531-548d9088d313\") " pod="openshift-monitoring/thanos-querier-57c946f66b-qt62h" Sep 10 12:19:23.749615 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.749534 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnvks\" (UniqueName: \"kubernetes.io/projected/6675777c-d2f9-4754-8531-548d9088d313-kube-api-access-tnvks\") pod \"thanos-querier-57c946f66b-qt62h\" (UID: \"6675777c-d2f9-4754-8531-548d9088d313\") " pod="openshift-monitoring/thanos-querier-57c946f66b-qt62h" Sep 10 12:19:23.749615 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.749568 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6675777c-d2f9-4754-8531-548d9088d313-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-57c946f66b-qt62h\" (UID: \"6675777c-d2f9-4754-8531-548d9088d313\") " pod="openshift-monitoring/thanos-querier-57c946f66b-qt62h" Sep 10 12:19:23.749615 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.749611 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6675777c-d2f9-4754-8531-548d9088d313-metrics-client-ca\") pod \"thanos-querier-57c946f66b-qt62h\" (UID: \"6675777c-d2f9-4754-8531-548d9088d313\") " pod="openshift-monitoring/thanos-querier-57c946f66b-qt62h" Sep 10 12:19:23.850878 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.850838 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6675777c-d2f9-4754-8531-548d9088d313-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-57c946f66b-qt62h\" (UID: \"6675777c-d2f9-4754-8531-548d9088d313\") " pod="openshift-monitoring/thanos-querier-57c946f66b-qt62h" Sep 10 12:19:23.851324 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.850898 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6675777c-d2f9-4754-8531-548d9088d313-secret-grpc-tls\") pod \"thanos-querier-57c946f66b-qt62h\" (UID: \"6675777c-d2f9-4754-8531-548d9088d313\") " pod="openshift-monitoring/thanos-querier-57c946f66b-qt62h" Sep 10 12:19:23.851324 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.850928 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnvks\" (UniqueName: \"kubernetes.io/projected/6675777c-d2f9-4754-8531-548d9088d313-kube-api-access-tnvks\") pod \"thanos-querier-57c946f66b-qt62h\" (UID: \"6675777c-d2f9-4754-8531-548d9088d313\") " pod="openshift-monitoring/thanos-querier-57c946f66b-qt62h" Sep 10 12:19:23.851324 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.850955 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6675777c-d2f9-4754-8531-548d9088d313-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-57c946f66b-qt62h\" (UID: \"6675777c-d2f9-4754-8531-548d9088d313\") " pod="openshift-monitoring/thanos-querier-57c946f66b-qt62h" Sep 10 12:19:23.851324 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.851006 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6675777c-d2f9-4754-8531-548d9088d313-metrics-client-ca\") pod \"thanos-querier-57c946f66b-qt62h\" (UID: \"6675777c-d2f9-4754-8531-548d9088d313\") " pod="openshift-monitoring/thanos-querier-57c946f66b-qt62h" Sep 10 12:19:23.851324 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.851062 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6675777c-d2f9-4754-8531-548d9088d313-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-57c946f66b-qt62h\" (UID: \"6675777c-d2f9-4754-8531-548d9088d313\") " pod="openshift-monitoring/thanos-querier-57c946f66b-qt62h" Sep 10 12:19:23.851324 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.851089 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6675777c-d2f9-4754-8531-548d9088d313-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-57c946f66b-qt62h\" (UID: \"6675777c-d2f9-4754-8531-548d9088d313\") " pod="openshift-monitoring/thanos-querier-57c946f66b-qt62h" Sep 10 12:19:23.851324 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.851115 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6675777c-d2f9-4754-8531-548d9088d313-secret-thanos-querier-tls\") pod \"thanos-querier-57c946f66b-qt62h\" (UID: \"6675777c-d2f9-4754-8531-548d9088d313\") " pod="openshift-monitoring/thanos-querier-57c946f66b-qt62h" Sep 10 12:19:23.853317 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.853208 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6675777c-d2f9-4754-8531-548d9088d313-metrics-client-ca\") pod \"thanos-querier-57c946f66b-qt62h\" (UID: \"6675777c-d2f9-4754-8531-548d9088d313\") " pod="openshift-monitoring/thanos-querier-57c946f66b-qt62h" Sep 10 12:19:23.854075 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.854045 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6675777c-d2f9-4754-8531-548d9088d313-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-57c946f66b-qt62h\" (UID: \"6675777c-d2f9-4754-8531-548d9088d313\") " pod="openshift-monitoring/thanos-querier-57c946f66b-qt62h" Sep 10 12:19:23.854700 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.854671 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6675777c-d2f9-4754-8531-548d9088d313-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-57c946f66b-qt62h\" (UID: \"6675777c-d2f9-4754-8531-548d9088d313\") " pod="openshift-monitoring/thanos-querier-57c946f66b-qt62h" Sep 10 12:19:23.855494 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.855468 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6675777c-d2f9-4754-8531-548d9088d313-secret-grpc-tls\") pod \"thanos-querier-57c946f66b-qt62h\" (UID: \"6675777c-d2f9-4754-8531-548d9088d313\") " pod="openshift-monitoring/thanos-querier-57c946f66b-qt62h" Sep 10 12:19:23.856233 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.856190 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6675777c-d2f9-4754-8531-548d9088d313-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-57c946f66b-qt62h\" (UID: \"6675777c-d2f9-4754-8531-548d9088d313\") " pod="openshift-monitoring/thanos-querier-57c946f66b-qt62h" Sep 10 12:19:23.856898 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.856704 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6675777c-d2f9-4754-8531-548d9088d313-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-57c946f66b-qt62h\" (UID: \"6675777c-d2f9-4754-8531-548d9088d313\") " pod="openshift-monitoring/thanos-querier-57c946f66b-qt62h" Sep 10 12:19:23.856898 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.856829 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6675777c-d2f9-4754-8531-548d9088d313-secret-thanos-querier-tls\") pod \"thanos-querier-57c946f66b-qt62h\" (UID: \"6675777c-d2f9-4754-8531-548d9088d313\") " pod="openshift-monitoring/thanos-querier-57c946f66b-qt62h" Sep 10 12:19:23.862668 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.862640 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnvks\" (UniqueName: \"kubernetes.io/projected/6675777c-d2f9-4754-8531-548d9088d313-kube-api-access-tnvks\") pod \"thanos-querier-57c946f66b-qt62h\" (UID: \"6675777c-d2f9-4754-8531-548d9088d313\") " pod="openshift-monitoring/thanos-querier-57c946f66b-qt62h" Sep 10 12:19:23.966427 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.966392 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-57c946f66b-qt62h" Sep 10 12:19:23.991696 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.991658 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-krrfz" event={"ID":"226b1299-cb0c-4568-8ae4-b3d955914d15","Type":"ContainerStarted","Data":"d61f2d393602d3aea330ec9e2893af46a8d48cd4888a4aeb0570b8a3790e813a"} Sep 10 12:19:23.994523 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.994453 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-wxwzn" event={"ID":"464a50e2-1786-451d-b7c5-567d20b5972f","Type":"ContainerStarted","Data":"523ec19733f28f2ff8aa5994ac4ace5dcfe772ad7fe119e624e9276046bbc0b6"} Sep 10 12:19:23.997081 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:23.996718 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b6452" event={"ID":"807c357a-a718-4e5e-8af5-29f075c8c276","Type":"ContainerStarted","Data":"5139dbc1f88b36497e1d1c7f9d50d294aae19da0334316dd7133d62feb6473b4"} Sep 10 12:19:24.069370 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:24.069323 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-wxwzn" podStartSLOduration=1.545607199 podStartE2EDuration="3.069303472s" totalImagesPullingTime="1.523696273s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:19:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:19:24.028344948 +0000 UTC m=+84.035661333" watchObservedRunningTime="2026-09-10 12:19:24.069303472 +0000 UTC m=+84.076619860" Sep 10 12:19:24.132295 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:24.132220 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 12:19:24.136221 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:19:24.136194 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1f6afd1_7472_4c06_8651_f15801fab26d.slice/crio-0f3e86a8631ab20ed0780a713a19da26f4a1b77142d993f65271b7c80fb05d62 WatchSource:0}: Error finding container 0f3e86a8631ab20ed0780a713a19da26f4a1b77142d993f65271b7c80fb05d62: Status 404 returned error can't find the container with id 0f3e86a8631ab20ed0780a713a19da26f4a1b77142d993f65271b7c80fb05d62 Sep 10 12:19:24.265261 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:24.265215 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-57c946f66b-qt62h"] Sep 10 12:19:24.268387 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:19:24.268360 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6675777c_d2f9_4754_8531_548d9088d313.slice/crio-8ca49c93398f4036f71112a0295742acae7e6ab9d53a7f0e25fa6878dc2e2487 WatchSource:0}: Error finding container 8ca49c93398f4036f71112a0295742acae7e6ab9d53a7f0e25fa6878dc2e2487: Status 404 returned error can't find the container with id 8ca49c93398f4036f71112a0295742acae7e6ab9d53a7f0e25fa6878dc2e2487 Sep 10 12:19:25.004098 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:25.004058 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-krrfz" event={"ID":"226b1299-cb0c-4568-8ae4-b3d955914d15","Type":"ContainerStarted","Data":"a1862601617b56a8d909232882491cb6699be96b4c5f367351b045aaf8680d69"} Sep 10 12:19:25.004553 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:25.004129 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-krrfz" event={"ID":"226b1299-cb0c-4568-8ae4-b3d955914d15","Type":"ContainerStarted","Data":"2ecfdedc957e2f8f7b316d6755b8c76c80b52e0784a95c339dd689a2dd71b72b"} Sep 10 12:19:25.006029 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:25.005998 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1f6afd1-7472-4c06-8651-f15801fab26d","Type":"ContainerStarted","Data":"0f3e86a8631ab20ed0780a713a19da26f4a1b77142d993f65271b7c80fb05d62"} Sep 10 12:19:25.008097 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:25.007961 2580 generic.go:356] "Generic (PLEG): container finished" podID="807c357a-a718-4e5e-8af5-29f075c8c276" containerID="5139dbc1f88b36497e1d1c7f9d50d294aae19da0334316dd7133d62feb6473b4" exitCode=0 Sep 10 12:19:25.008097 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:25.008041 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b6452" event={"ID":"807c357a-a718-4e5e-8af5-29f075c8c276","Type":"ContainerDied","Data":"5139dbc1f88b36497e1d1c7f9d50d294aae19da0334316dd7133d62feb6473b4"} Sep 10 12:19:25.009889 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:25.009798 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-57c946f66b-qt62h" event={"ID":"6675777c-d2f9-4754-8531-548d9088d313","Type":"ContainerStarted","Data":"8ca49c93398f4036f71112a0295742acae7e6ab9d53a7f0e25fa6878dc2e2487"} Sep 10 12:19:25.027846 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:25.026421 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-krrfz" podStartSLOduration=2.440944901 podStartE2EDuration="4.026402033s" totalImagesPullingTime="1.585457132s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:19:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:19:25.02604055 +0000 UTC m=+85.033356927" watchObservedRunningTime="2026-09-10 12:19:25.026402033 +0000 UTC m=+85.033718418" Sep 10 12:19:26.018868 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.018828 2580 generic.go:356] "Generic (PLEG): container finished" podID="f1f6afd1-7472-4c06-8651-f15801fab26d" containerID="666f47ea3575704231d421ff04711553f473eb12c8de77a5072eaad773f45215" exitCode=0 Sep 10 12:19:26.019334 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.018894 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1f6afd1-7472-4c06-8651-f15801fab26d","Type":"ContainerDied","Data":"666f47ea3575704231d421ff04711553f473eb12c8de77a5072eaad773f45215"} Sep 10 12:19:26.021694 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.021666 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b6452" event={"ID":"807c357a-a718-4e5e-8af5-29f075c8c276","Type":"ContainerStarted","Data":"5a49d849496d66cb90242c164338b173c8779a3a254fa5603a603407403effe4"} Sep 10 12:19:26.021798 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.021722 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b6452" event={"ID":"807c357a-a718-4e5e-8af5-29f075c8c276","Type":"ContainerStarted","Data":"4c4ed264154f9bb9bb1acf2ed91e4cc9e613dcc7c35b411c3a6d1ff354bce393"} Sep 10 12:19:26.125316 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.125268 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-b6452" podStartSLOduration=3.192248951 podStartE2EDuration="5.125251944s" totalImagesPullingTime="1.845610085s" totalInitContainerRuntime="87.392908ms" isStatefulPod=true podCreationTimestamp="2026-09-10 12:19:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:19:26.124072743 +0000 UTC m=+86.131389125" watchObservedRunningTime="2026-09-10 12:19:26.125251944 +0000 UTC m=+86.132568327" Sep 10 12:19:26.162248 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.162223 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5bbdc6f8cb-sck44"] Sep 10 12:19:26.165217 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.165201 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5bbdc6f8cb-sck44" Sep 10 12:19:26.168309 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.168287 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-sjnl9\"" Sep 10 12:19:26.168309 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.168297 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Sep 10 12:19:26.168484 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.168333 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 10 12:19:26.168484 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.168418 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Sep 10 12:19:26.168484 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.168430 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-9f1tcb3pfj5nm\"" Sep 10 12:19:26.168703 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.168687 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Sep 10 12:19:26.179435 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.179416 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5bbdc6f8cb-sck44"] Sep 10 12:19:26.278355 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.278332 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a17142e5-cea9-4e49-a808-f1310b79ea4c-secret-metrics-server-client-certs\") pod \"metrics-server-5bbdc6f8cb-sck44\" (UID: \"a17142e5-cea9-4e49-a808-f1310b79ea4c\") " pod="openshift-monitoring/metrics-server-5bbdc6f8cb-sck44" Sep 10 12:19:26.278441 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.278367 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a17142e5-cea9-4e49-a808-f1310b79ea4c-secret-metrics-server-tls\") pod \"metrics-server-5bbdc6f8cb-sck44\" (UID: \"a17142e5-cea9-4e49-a808-f1310b79ea4c\") " pod="openshift-monitoring/metrics-server-5bbdc6f8cb-sck44" Sep 10 12:19:26.278441 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.278400 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a17142e5-cea9-4e49-a808-f1310b79ea4c-metrics-server-audit-profiles\") pod \"metrics-server-5bbdc6f8cb-sck44\" (UID: \"a17142e5-cea9-4e49-a808-f1310b79ea4c\") " pod="openshift-monitoring/metrics-server-5bbdc6f8cb-sck44" Sep 10 12:19:26.278525 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.278462 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a17142e5-cea9-4e49-a808-f1310b79ea4c-audit-log\") pod \"metrics-server-5bbdc6f8cb-sck44\" (UID: \"a17142e5-cea9-4e49-a808-f1310b79ea4c\") " pod="openshift-monitoring/metrics-server-5bbdc6f8cb-sck44" Sep 10 12:19:26.278525 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.278488 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a17142e5-cea9-4e49-a808-f1310b79ea4c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5bbdc6f8cb-sck44\" (UID: \"a17142e5-cea9-4e49-a808-f1310b79ea4c\") " pod="openshift-monitoring/metrics-server-5bbdc6f8cb-sck44" Sep 10 12:19:26.278634 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.278539 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a17142e5-cea9-4e49-a808-f1310b79ea4c-client-ca-bundle\") pod \"metrics-server-5bbdc6f8cb-sck44\" (UID: \"a17142e5-cea9-4e49-a808-f1310b79ea4c\") " pod="openshift-monitoring/metrics-server-5bbdc6f8cb-sck44" Sep 10 12:19:26.278634 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.278604 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfv79\" (UniqueName: \"kubernetes.io/projected/a17142e5-cea9-4e49-a808-f1310b79ea4c-kube-api-access-rfv79\") pod \"metrics-server-5bbdc6f8cb-sck44\" (UID: \"a17142e5-cea9-4e49-a808-f1310b79ea4c\") " pod="openshift-monitoring/metrics-server-5bbdc6f8cb-sck44" Sep 10 12:19:26.379604 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.379572 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a17142e5-cea9-4e49-a808-f1310b79ea4c-audit-log\") pod \"metrics-server-5bbdc6f8cb-sck44\" (UID: \"a17142e5-cea9-4e49-a808-f1310b79ea4c\") " pod="openshift-monitoring/metrics-server-5bbdc6f8cb-sck44" Sep 10 12:19:26.379779 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.379610 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a17142e5-cea9-4e49-a808-f1310b79ea4c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5bbdc6f8cb-sck44\" (UID: \"a17142e5-cea9-4e49-a808-f1310b79ea4c\") " pod="openshift-monitoring/metrics-server-5bbdc6f8cb-sck44" Sep 10 12:19:26.379779 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.379643 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a17142e5-cea9-4e49-a808-f1310b79ea4c-client-ca-bundle\") pod \"metrics-server-5bbdc6f8cb-sck44\" (UID: \"a17142e5-cea9-4e49-a808-f1310b79ea4c\") " pod="openshift-monitoring/metrics-server-5bbdc6f8cb-sck44" Sep 10 12:19:26.379779 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.379708 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfv79\" (UniqueName: \"kubernetes.io/projected/a17142e5-cea9-4e49-a808-f1310b79ea4c-kube-api-access-rfv79\") pod \"metrics-server-5bbdc6f8cb-sck44\" (UID: \"a17142e5-cea9-4e49-a808-f1310b79ea4c\") " pod="openshift-monitoring/metrics-server-5bbdc6f8cb-sck44" Sep 10 12:19:26.379963 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.379780 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a17142e5-cea9-4e49-a808-f1310b79ea4c-secret-metrics-server-client-certs\") pod \"metrics-server-5bbdc6f8cb-sck44\" (UID: \"a17142e5-cea9-4e49-a808-f1310b79ea4c\") " pod="openshift-monitoring/metrics-server-5bbdc6f8cb-sck44" Sep 10 12:19:26.379963 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.379880 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a17142e5-cea9-4e49-a808-f1310b79ea4c-secret-metrics-server-tls\") pod \"metrics-server-5bbdc6f8cb-sck44\" (UID: \"a17142e5-cea9-4e49-a808-f1310b79ea4c\") " pod="openshift-monitoring/metrics-server-5bbdc6f8cb-sck44" Sep 10 12:19:26.379963 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.379926 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a17142e5-cea9-4e49-a808-f1310b79ea4c-metrics-server-audit-profiles\") pod \"metrics-server-5bbdc6f8cb-sck44\" (UID: \"a17142e5-cea9-4e49-a808-f1310b79ea4c\") " pod="openshift-monitoring/metrics-server-5bbdc6f8cb-sck44" Sep 10 12:19:26.380102 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.379989 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a17142e5-cea9-4e49-a808-f1310b79ea4c-audit-log\") pod \"metrics-server-5bbdc6f8cb-sck44\" (UID: \"a17142e5-cea9-4e49-a808-f1310b79ea4c\") " pod="openshift-monitoring/metrics-server-5bbdc6f8cb-sck44" Sep 10 12:19:26.380632 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.380608 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a17142e5-cea9-4e49-a808-f1310b79ea4c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5bbdc6f8cb-sck44\" (UID: \"a17142e5-cea9-4e49-a808-f1310b79ea4c\") " pod="openshift-monitoring/metrics-server-5bbdc6f8cb-sck44" Sep 10 12:19:26.380979 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.380954 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a17142e5-cea9-4e49-a808-f1310b79ea4c-metrics-server-audit-profiles\") pod \"metrics-server-5bbdc6f8cb-sck44\" (UID: \"a17142e5-cea9-4e49-a808-f1310b79ea4c\") " pod="openshift-monitoring/metrics-server-5bbdc6f8cb-sck44" Sep 10 12:19:26.382402 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.382381 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a17142e5-cea9-4e49-a808-f1310b79ea4c-client-ca-bundle\") pod \"metrics-server-5bbdc6f8cb-sck44\" (UID: \"a17142e5-cea9-4e49-a808-f1310b79ea4c\") " pod="openshift-monitoring/metrics-server-5bbdc6f8cb-sck44" Sep 10 12:19:26.382503 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.382479 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a17142e5-cea9-4e49-a808-f1310b79ea4c-secret-metrics-server-tls\") pod \"metrics-server-5bbdc6f8cb-sck44\" (UID: \"a17142e5-cea9-4e49-a808-f1310b79ea4c\") " pod="openshift-monitoring/metrics-server-5bbdc6f8cb-sck44" Sep 10 12:19:26.382559 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.382513 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a17142e5-cea9-4e49-a808-f1310b79ea4c-secret-metrics-server-client-certs\") pod \"metrics-server-5bbdc6f8cb-sck44\" (UID: \"a17142e5-cea9-4e49-a808-f1310b79ea4c\") " pod="openshift-monitoring/metrics-server-5bbdc6f8cb-sck44" Sep 10 12:19:26.397122 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.397102 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfv79\" (UniqueName: \"kubernetes.io/projected/a17142e5-cea9-4e49-a808-f1310b79ea4c-kube-api-access-rfv79\") pod \"metrics-server-5bbdc6f8cb-sck44\" (UID: \"a17142e5-cea9-4e49-a808-f1310b79ea4c\") " pod="openshift-monitoring/metrics-server-5bbdc6f8cb-sck44" Sep 10 12:19:26.407174 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.407151 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-d4hlf"] Sep 10 12:19:26.411397 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.411379 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-d4hlf" Sep 10 12:19:26.415597 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.415578 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Sep 10 12:19:26.415691 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.415580 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-cf5z9\"" Sep 10 12:19:26.447552 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.447523 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-d4hlf"] Sep 10 12:19:26.480379 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.480356 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5bbdc6f8cb-sck44" Sep 10 12:19:26.581077 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.581051 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/41360c8c-a2f7-4c31-84c5-c92265753526-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-d4hlf\" (UID: \"41360c8c-a2f7-4c31-84c5-c92265753526\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-d4hlf" Sep 10 12:19:26.581222 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.581172 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5t7c\" (UniqueName: \"kubernetes.io/projected/41360c8c-a2f7-4c31-84c5-c92265753526-kube-api-access-f5t7c\") pod \"monitoring-plugin-85bbbbbf4-d4hlf\" (UID: \"41360c8c-a2f7-4c31-84c5-c92265753526\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-d4hlf" Sep 10 12:19:26.682075 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.682037 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5t7c\" (UniqueName: \"kubernetes.io/projected/41360c8c-a2f7-4c31-84c5-c92265753526-kube-api-access-f5t7c\") pod \"monitoring-plugin-85bbbbbf4-d4hlf\" (UID: \"41360c8c-a2f7-4c31-84c5-c92265753526\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-d4hlf" Sep 10 12:19:26.682258 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.682097 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/41360c8c-a2f7-4c31-84c5-c92265753526-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-d4hlf\" (UID: \"41360c8c-a2f7-4c31-84c5-c92265753526\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-d4hlf" Sep 10 12:19:26.682258 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:19:26.682232 2580 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Sep 10 12:19:26.682374 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:19:26.682300 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41360c8c-a2f7-4c31-84c5-c92265753526-monitoring-plugin-cert podName:41360c8c-a2f7-4c31-84c5-c92265753526 nodeName:}" failed. No retries permitted until 2026-09-10 12:19:27.182280153 +0000 UTC m=+87.189596514 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/41360c8c-a2f7-4c31-84c5-c92265753526-monitoring-plugin-cert") pod "monitoring-plugin-85bbbbbf4-d4hlf" (UID: "41360c8c-a2f7-4c31-84c5-c92265753526") : secret "monitoring-plugin-cert" not found Sep 10 12:19:26.694182 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.694153 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5t7c\" (UniqueName: \"kubernetes.io/projected/41360c8c-a2f7-4c31-84c5-c92265753526-kube-api-access-f5t7c\") pod \"monitoring-plugin-85bbbbbf4-d4hlf\" (UID: \"41360c8c-a2f7-4c31-84c5-c92265753526\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-d4hlf" Sep 10 12:19:26.760087 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:26.760057 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5bbdc6f8cb-sck44"] Sep 10 12:19:26.763452 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:19:26.763417 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda17142e5_cea9_4e49_a808_f1310b79ea4c.slice/crio-f42818defd007f6d47def322912dea4b95a6c797038da5f3dbf1bad7bd14d1d3 WatchSource:0}: Error finding container f42818defd007f6d47def322912dea4b95a6c797038da5f3dbf1bad7bd14d1d3: Status 404 returned error can't find the container with id f42818defd007f6d47def322912dea4b95a6c797038da5f3dbf1bad7bd14d1d3 Sep 10 12:19:27.031401 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:27.031269 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5bbdc6f8cb-sck44" event={"ID":"a17142e5-cea9-4e49-a808-f1310b79ea4c","Type":"ContainerStarted","Data":"f42818defd007f6d47def322912dea4b95a6c797038da5f3dbf1bad7bd14d1d3"} Sep 10 12:19:27.037358 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:27.036852 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-57c946f66b-qt62h" event={"ID":"6675777c-d2f9-4754-8531-548d9088d313","Type":"ContainerStarted","Data":"099ba98d233a1bb8d9447cf0c403fc9cec8c7fc9a6923ecd8b08bbc5a9d4025f"} Sep 10 12:19:27.037358 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:27.036945 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-57c946f66b-qt62h" event={"ID":"6675777c-d2f9-4754-8531-548d9088d313","Type":"ContainerStarted","Data":"90b2ef3c7e7f1d899a336c97357fee7205293fe988248662b45b50ff324f1a10"} Sep 10 12:19:27.037358 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:27.036988 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-57c946f66b-qt62h" event={"ID":"6675777c-d2f9-4754-8531-548d9088d313","Type":"ContainerStarted","Data":"3e1475b71f653cabba796bcfa13f1dda6435139aec2854ba89f9654ec6287033"} Sep 10 12:19:27.188270 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:27.187951 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/41360c8c-a2f7-4c31-84c5-c92265753526-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-d4hlf\" (UID: \"41360c8c-a2f7-4c31-84c5-c92265753526\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-d4hlf" Sep 10 12:19:27.191139 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:27.191090 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/41360c8c-a2f7-4c31-84c5-c92265753526-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-d4hlf\" (UID: \"41360c8c-a2f7-4c31-84c5-c92265753526\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-d4hlf" Sep 10 12:19:27.320495 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:27.320419 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-d4hlf" Sep 10 12:19:27.892339 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:27.892315 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-d4hlf"] Sep 10 12:19:27.896806 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:19:27.896776 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41360c8c_a2f7_4c31_84c5_c92265753526.slice/crio-71f5a8fe385e360468e9dd64f0ffad0a586196580f1f8345b85185b1ae2e160b WatchSource:0}: Error finding container 71f5a8fe385e360468e9dd64f0ffad0a586196580f1f8345b85185b1ae2e160b: Status 404 returned error can't find the container with id 71f5a8fe385e360468e9dd64f0ffad0a586196580f1f8345b85185b1ae2e160b Sep 10 12:19:27.982315 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:27.982287 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 12:19:27.988701 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:27.988091 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:27.992671 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:27.992479 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 10 12:19:27.993658 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:27.992928 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 10 12:19:27.993658 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:27.993138 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 10 12:19:27.993658 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:27.993155 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-xnws5\"" Sep 10 12:19:27.993658 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:27.993321 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 10 12:19:27.993658 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:27.993368 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 10 12:19:27.993658 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:27.993378 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 10 12:19:27.993658 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:27.993508 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 10 12:19:27.993658 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:27.993582 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 10 12:19:27.993658 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:27.993603 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 10 12:19:27.994799 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:27.994245 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 10 12:19:27.994799 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:27.994385 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 10 12:19:27.994799 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:27.994506 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2mdtpbph0tvud\"" Sep 10 12:19:27.994799 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:27.994683 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 10 12:19:27.995693 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:27.995472 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 10 12:19:27.997302 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:27.996943 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 10 12:19:28.009825 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.007822 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 12:19:28.043997 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.043965 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1f6afd1-7472-4c06-8651-f15801fab26d","Type":"ContainerStarted","Data":"1552cdc2107704f5108f6e6a9f1fe93baecd7545169aa9161a06bd6f1847ef01"} Sep 10 12:19:28.044397 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.044051 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1f6afd1-7472-4c06-8651-f15801fab26d","Type":"ContainerStarted","Data":"29e275eaef6af0b0ae2b7d483647adce32fc5193f155b8da9befda4d49786cf2"} Sep 10 12:19:28.044397 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.044098 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1f6afd1-7472-4c06-8651-f15801fab26d","Type":"ContainerStarted","Data":"c1b1e5acce3315b9d0538539d66bf8d1f7b9d943ce16ca90bad0ae70cf58e620"} Sep 10 12:19:28.044397 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.044136 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1f6afd1-7472-4c06-8651-f15801fab26d","Type":"ContainerStarted","Data":"e57e96917a7f98e2bb13303d89d940783b3da81aa4773c661f08e3c346e6d68b"} Sep 10 12:19:28.050197 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.049388 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-57c946f66b-qt62h" event={"ID":"6675777c-d2f9-4754-8531-548d9088d313","Type":"ContainerStarted","Data":"6cdb050c50d4fbf8b0bea6c09947c7f8ab2834b95607b44955118b71becf9f8e"} Sep 10 12:19:28.050197 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.049515 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-57c946f66b-qt62h" Sep 10 12:19:28.050197 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.049559 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-57c946f66b-qt62h" event={"ID":"6675777c-d2f9-4754-8531-548d9088d313","Type":"ContainerStarted","Data":"c4da66978da7337b2be4cf3865e75eb4bf9bd1728ffc1bd7aff5440924566852"} Sep 10 12:19:28.050197 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.049603 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-57c946f66b-qt62h" event={"ID":"6675777c-d2f9-4754-8531-548d9088d313","Type":"ContainerStarted","Data":"ab3520e6a6ac971e851f70598dcff2b220f5b816302832ca3f12d5b43e1c0d9f"} Sep 10 12:19:28.051183 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.051159 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-d4hlf" event={"ID":"41360c8c-a2f7-4c31-84c5-c92265753526","Type":"ContainerStarted","Data":"71f5a8fe385e360468e9dd64f0ffad0a586196580f1f8345b85185b1ae2e160b"} Sep 10 12:19:28.095975 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.095943 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/98426a97-564b-45a6-b15e-6a018de393d5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.096090 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.095994 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/98426a97-564b-45a6-b15e-6a018de393d5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.096090 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.096021 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/98426a97-564b-45a6-b15e-6a018de393d5-web-config\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.096090 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.096070 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ps9gl\" (UniqueName: \"kubernetes.io/projected/98426a97-564b-45a6-b15e-6a018de393d5-kube-api-access-ps9gl\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.096211 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.096125 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/98426a97-564b-45a6-b15e-6a018de393d5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.096211 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.096161 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/98426a97-564b-45a6-b15e-6a018de393d5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.096211 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.096188 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/98426a97-564b-45a6-b15e-6a018de393d5-config\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.096396 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.096212 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98426a97-564b-45a6-b15e-6a018de393d5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.096396 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.096248 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/98426a97-564b-45a6-b15e-6a018de393d5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.096396 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.096285 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/98426a97-564b-45a6-b15e-6a018de393d5-config-out\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.096396 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.096363 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/98426a97-564b-45a6-b15e-6a018de393d5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.096514 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.096396 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/98426a97-564b-45a6-b15e-6a018de393d5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.096514 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.096429 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98426a97-564b-45a6-b15e-6a018de393d5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.096514 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.096497 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98426a97-564b-45a6-b15e-6a018de393d5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.096621 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.096525 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/98426a97-564b-45a6-b15e-6a018de393d5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.096621 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.096547 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/98426a97-564b-45a6-b15e-6a018de393d5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.096621 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.096595 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/98426a97-564b-45a6-b15e-6a018de393d5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.096621 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.096615 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/98426a97-564b-45a6-b15e-6a018de393d5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.096793 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.096641 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/98426a97-564b-45a6-b15e-6a018de393d5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.096793 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.096676 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/98426a97-564b-45a6-b15e-6a018de393d5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.165951 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.165896 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-57c946f66b-qt62h" podStartSLOduration=1.984790552 podStartE2EDuration="5.165878687s" totalImagesPullingTime="3.181088135s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:19:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:19:28.159523415 +0000 UTC m=+88.166839842" watchObservedRunningTime="2026-09-10 12:19:28.165878687 +0000 UTC m=+88.173195060" Sep 10 12:19:28.197337 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.197307 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/98426a97-564b-45a6-b15e-6a018de393d5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.198190 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.198136 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/98426a97-564b-45a6-b15e-6a018de393d5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.198441 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.198401 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/98426a97-564b-45a6-b15e-6a018de393d5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.199732 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.199639 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/98426a97-564b-45a6-b15e-6a018de393d5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.199732 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.199704 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/98426a97-564b-45a6-b15e-6a018de393d5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.199974 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.199786 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/98426a97-564b-45a6-b15e-6a018de393d5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.199974 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.199821 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/98426a97-564b-45a6-b15e-6a018de393d5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.199974 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.199850 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/98426a97-564b-45a6-b15e-6a018de393d5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.199974 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.199897 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/98426a97-564b-45a6-b15e-6a018de393d5-web-config\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.199974 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.199927 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ps9gl\" (UniqueName: \"kubernetes.io/projected/98426a97-564b-45a6-b15e-6a018de393d5-kube-api-access-ps9gl\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.199974 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.199971 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/98426a97-564b-45a6-b15e-6a018de393d5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.200385 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.200001 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/98426a97-564b-45a6-b15e-6a018de393d5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.200385 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.200042 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/98426a97-564b-45a6-b15e-6a018de393d5-config\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.200385 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.200066 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98426a97-564b-45a6-b15e-6a018de393d5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.200385 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.200091 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/98426a97-564b-45a6-b15e-6a018de393d5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.200385 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.200114 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/98426a97-564b-45a6-b15e-6a018de393d5-config-out\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.200385 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.200208 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/98426a97-564b-45a6-b15e-6a018de393d5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.200385 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.200240 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/98426a97-564b-45a6-b15e-6a018de393d5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.200385 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.200279 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98426a97-564b-45a6-b15e-6a018de393d5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.200385 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.200348 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98426a97-564b-45a6-b15e-6a018de393d5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.200385 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.200379 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/98426a97-564b-45a6-b15e-6a018de393d5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.200839 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.200405 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/98426a97-564b-45a6-b15e-6a018de393d5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.200839 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.200648 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/98426a97-564b-45a6-b15e-6a018de393d5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.202053 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.201764 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/98426a97-564b-45a6-b15e-6a018de393d5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.203184 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.202944 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/98426a97-564b-45a6-b15e-6a018de393d5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.203803 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.203770 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98426a97-564b-45a6-b15e-6a018de393d5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.203906 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.203807 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98426a97-564b-45a6-b15e-6a018de393d5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.203906 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.203813 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98426a97-564b-45a6-b15e-6a018de393d5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.205177 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.205136 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/98426a97-564b-45a6-b15e-6a018de393d5-config-out\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.206174 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.205812 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/98426a97-564b-45a6-b15e-6a018de393d5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.206174 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.205928 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/98426a97-564b-45a6-b15e-6a018de393d5-config\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.206330 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.206214 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/98426a97-564b-45a6-b15e-6a018de393d5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.206383 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.206325 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/98426a97-564b-45a6-b15e-6a018de393d5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.206383 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.206154 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/98426a97-564b-45a6-b15e-6a018de393d5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.206687 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.206665 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/98426a97-564b-45a6-b15e-6a018de393d5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.207088 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.207064 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/98426a97-564b-45a6-b15e-6a018de393d5-web-config\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.207348 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.207332 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/98426a97-564b-45a6-b15e-6a018de393d5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.207393 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.207352 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/98426a97-564b-45a6-b15e-6a018de393d5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.208508 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.208487 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/98426a97-564b-45a6-b15e-6a018de393d5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.211826 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.211808 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ps9gl\" (UniqueName: \"kubernetes.io/projected/98426a97-564b-45a6-b15e-6a018de393d5-kube-api-access-ps9gl\") pod \"prometheus-k8s-0\" (UID: \"98426a97-564b-45a6-b15e-6a018de393d5\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.301108 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.301071 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:28.606421 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:28.606320 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 12:19:28.843396 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:19:28.843361 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98426a97_564b_45a6_b15e_6a018de393d5.slice/crio-9bdbd859811797aa3ebca05ce37adb102404ce49e27ece866f3b8cd44ff3a3ca WatchSource:0}: Error finding container 9bdbd859811797aa3ebca05ce37adb102404ce49e27ece866f3b8cd44ff3a3ca: Status 404 returned error can't find the container with id 9bdbd859811797aa3ebca05ce37adb102404ce49e27ece866f3b8cd44ff3a3ca Sep 10 12:19:29.060515 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:29.060484 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1f6afd1-7472-4c06-8651-f15801fab26d","Type":"ContainerStarted","Data":"22a208b78e56c45ce25c807ac06bcb45b18668419b5267df701257c2b63d3c9f"} Sep 10 12:19:29.060960 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:29.060562 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1f6afd1-7472-4c06-8651-f15801fab26d","Type":"ContainerStarted","Data":"c1128e7c0a94b479216c12ba980080d87dd25a2eb3da20f29e08a1806c1d5457"} Sep 10 12:19:29.062464 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:29.062410 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5bbdc6f8cb-sck44" event={"ID":"a17142e5-cea9-4e49-a808-f1310b79ea4c","Type":"ContainerStarted","Data":"768f0eca4902224ecfab2fb24e0f5a3fa90c414cdda41083c7e6902d9c87b001"} Sep 10 12:19:29.064160 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:29.064133 2580 generic.go:356] "Generic (PLEG): container finished" podID="98426a97-564b-45a6-b15e-6a018de393d5" containerID="8662666cc38b2149ea6ca5f08920a6ca08e7aad638ff959de4c9a8b2220f8d5b" exitCode=0 Sep 10 12:19:29.064330 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:29.064302 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"98426a97-564b-45a6-b15e-6a018de393d5","Type":"ContainerDied","Data":"8662666cc38b2149ea6ca5f08920a6ca08e7aad638ff959de4c9a8b2220f8d5b"} Sep 10 12:19:29.064448 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:29.064386 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"98426a97-564b-45a6-b15e-6a018de393d5","Type":"ContainerStarted","Data":"9bdbd859811797aa3ebca05ce37adb102404ce49e27ece866f3b8cd44ff3a3ca"} Sep 10 12:19:29.092873 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:29.092809 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.191540804 podStartE2EDuration="7.092790524s" totalImagesPullingTime="2.882344258s" totalInitContainerRuntime="18.905462ms" isStatefulPod=false podCreationTimestamp="2026-09-10 12:19:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:19:29.091237964 +0000 UTC m=+89.098554348" watchObservedRunningTime="2026-09-10 12:19:29.092790524 +0000 UTC m=+89.100106909" Sep 10 12:19:29.144016 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:29.143900 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5bbdc6f8cb-sck44" podStartSLOduration=1.022361685 podStartE2EDuration="3.143876943s" totalImagesPullingTime="2.121515258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:19:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:19:29.142664907 +0000 UTC m=+89.149981291" watchObservedRunningTime="2026-09-10 12:19:29.143876943 +0000 UTC m=+89.151193324" Sep 10 12:19:30.071141 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:30.071098 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-d4hlf" event={"ID":"41360c8c-a2f7-4c31-84c5-c92265753526","Type":"ContainerStarted","Data":"afcb039645abd71639993498bd40b041bcf4253a067c791c6a1e47adf9a5a303"} Sep 10 12:19:30.090118 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:30.090072 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-d4hlf" podStartSLOduration=2.56845115 podStartE2EDuration="4.090058151s" totalImagesPullingTime="1.521607001s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:19:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:19:30.088579158 +0000 UTC m=+90.095895547" watchObservedRunningTime="2026-09-10 12:19:30.090058151 +0000 UTC m=+90.097374534" Sep 10 12:19:31.076353 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:31.076318 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-d4hlf" Sep 10 12:19:31.081810 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:31.081787 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-d4hlf" Sep 10 12:19:33.091227 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:33.091189 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"98426a97-564b-45a6-b15e-6a018de393d5","Type":"ContainerStarted","Data":"3f99638e1522420eec500e89c2202410ef17e56e9052d621f871ca31a1ad44fa"} Sep 10 12:19:33.091655 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:33.091258 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"98426a97-564b-45a6-b15e-6a018de393d5","Type":"ContainerStarted","Data":"3dc89f8df9412b593b89436cc5c0785b306d74fc90d28c85511e651d2225318f"} Sep 10 12:19:33.091655 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:33.091287 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"98426a97-564b-45a6-b15e-6a018de393d5","Type":"ContainerStarted","Data":"b4a3e401cf55256aa152bcb832864739c1c062d00361941387f95f15d09b1cde"} Sep 10 12:19:33.091655 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:33.091310 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"98426a97-564b-45a6-b15e-6a018de393d5","Type":"ContainerStarted","Data":"1252e57d50a8127a644406e530e723bb6c74c77dd5866ee0b5c91c2d255e959a"} Sep 10 12:19:33.091655 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:33.091335 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"98426a97-564b-45a6-b15e-6a018de393d5","Type":"ContainerStarted","Data":"96decb8f89b77c1481298d5c148069aefb83a743f530531a3956786c1ddfb571"} Sep 10 12:19:33.091655 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:33.091361 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"98426a97-564b-45a6-b15e-6a018de393d5","Type":"ContainerStarted","Data":"504ec021387eaea6122bbd0e98e9a835bd63d56593be070b4a99854692a9f49f"} Sep 10 12:19:33.126375 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:33.126310 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.885379164 podStartE2EDuration="6.126288684s" totalImagesPullingTime="3.216425101s" totalInitContainerRuntime="24.484419ms" isStatefulPod=false podCreationTimestamp="2026-09-10 12:19:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:19:33.12351747 +0000 UTC m=+93.130833881" watchObservedRunningTime="2026-09-10 12:19:33.126288684 +0000 UTC m=+93.133605068" Sep 10 12:19:33.302227 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:33.302198 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:19:34.070515 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:34.070486 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-57c946f66b-qt62h" Sep 10 12:19:37.054893 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:37.054807 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b565596bf-z8wcz"] Sep 10 12:19:46.480476 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:46.480435 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5bbdc6f8cb-sck44" Sep 10 12:19:46.480951 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:19:46.480506 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5bbdc6f8cb-sck44" Sep 10 12:20:02.074195 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:02.074129 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6b565596bf-z8wcz" podUID="afdcb83e-8517-4833-af04-22feff9dc221" containerName="console" containerID="cri-o://3741c1eb9bd3cc9f4f42301d40d04e04aa31f353f3c62f50e3b48e2808480286" gracePeriod=15 Sep 10 12:20:02.205750 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:02.205721 2580 generic.go:356] "Generic (PLEG): container finished" podID="afdcb83e-8517-4833-af04-22feff9dc221" containerID="3741c1eb9bd3cc9f4f42301d40d04e04aa31f353f3c62f50e3b48e2808480286" exitCode=0 Sep 10 12:20:02.205914 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:02.205772 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b565596bf-z8wcz" event={"ID":"afdcb83e-8517-4833-af04-22feff9dc221","Type":"ContainerDied","Data":"3741c1eb9bd3cc9f4f42301d40d04e04aa31f353f3c62f50e3b48e2808480286"} Sep 10 12:20:02.348562 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:02.348506 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b565596bf-z8wcz" Sep 10 12:20:02.513691 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:02.513658 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/afdcb83e-8517-4833-af04-22feff9dc221-console-oauth-config\") pod \"afdcb83e-8517-4833-af04-22feff9dc221\" (UID: \"afdcb83e-8517-4833-af04-22feff9dc221\") " Sep 10 12:20:02.513691 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:02.513696 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/afdcb83e-8517-4833-af04-22feff9dc221-service-ca\") pod \"afdcb83e-8517-4833-af04-22feff9dc221\" (UID: \"afdcb83e-8517-4833-af04-22feff9dc221\") " Sep 10 12:20:02.513955 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:02.513722 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/afdcb83e-8517-4833-af04-22feff9dc221-console-serving-cert\") pod \"afdcb83e-8517-4833-af04-22feff9dc221\" (UID: \"afdcb83e-8517-4833-af04-22feff9dc221\") " Sep 10 12:20:02.513955 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:02.513750 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/afdcb83e-8517-4833-af04-22feff9dc221-console-config\") pod \"afdcb83e-8517-4833-af04-22feff9dc221\" (UID: \"afdcb83e-8517-4833-af04-22feff9dc221\") " Sep 10 12:20:02.513955 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:02.513793 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d8jnx\" (UniqueName: \"kubernetes.io/projected/afdcb83e-8517-4833-af04-22feff9dc221-kube-api-access-d8jnx\") pod \"afdcb83e-8517-4833-af04-22feff9dc221\" (UID: \"afdcb83e-8517-4833-af04-22feff9dc221\") " Sep 10 12:20:02.513955 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:02.513821 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/afdcb83e-8517-4833-af04-22feff9dc221-oauth-serving-cert\") pod \"afdcb83e-8517-4833-af04-22feff9dc221\" (UID: \"afdcb83e-8517-4833-af04-22feff9dc221\") " Sep 10 12:20:02.513955 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:02.513843 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afdcb83e-8517-4833-af04-22feff9dc221-trusted-ca-bundle\") pod \"afdcb83e-8517-4833-af04-22feff9dc221\" (UID: \"afdcb83e-8517-4833-af04-22feff9dc221\") " Sep 10 12:20:02.514286 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:02.514257 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afdcb83e-8517-4833-af04-22feff9dc221-console-config" (OuterVolumeSpecName: "console-config") pod "afdcb83e-8517-4833-af04-22feff9dc221" (UID: "afdcb83e-8517-4833-af04-22feff9dc221"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:20:02.514456 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:02.514429 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afdcb83e-8517-4833-af04-22feff9dc221-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "afdcb83e-8517-4833-af04-22feff9dc221" (UID: "afdcb83e-8517-4833-af04-22feff9dc221"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:20:02.514456 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:02.514438 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afdcb83e-8517-4833-af04-22feff9dc221-service-ca" (OuterVolumeSpecName: "service-ca") pod "afdcb83e-8517-4833-af04-22feff9dc221" (UID: "afdcb83e-8517-4833-af04-22feff9dc221"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:20:02.514574 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:02.514473 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afdcb83e-8517-4833-af04-22feff9dc221-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "afdcb83e-8517-4833-af04-22feff9dc221" (UID: "afdcb83e-8517-4833-af04-22feff9dc221"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:20:02.516161 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:02.516136 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afdcb83e-8517-4833-af04-22feff9dc221-kube-api-access-d8jnx" (OuterVolumeSpecName: "kube-api-access-d8jnx") pod "afdcb83e-8517-4833-af04-22feff9dc221" (UID: "afdcb83e-8517-4833-af04-22feff9dc221"). InnerVolumeSpecName "kube-api-access-d8jnx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:20:02.516231 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:02.516183 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afdcb83e-8517-4833-af04-22feff9dc221-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "afdcb83e-8517-4833-af04-22feff9dc221" (UID: "afdcb83e-8517-4833-af04-22feff9dc221"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:20:02.516267 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:02.516245 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afdcb83e-8517-4833-af04-22feff9dc221-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "afdcb83e-8517-4833-af04-22feff9dc221" (UID: "afdcb83e-8517-4833-af04-22feff9dc221"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:20:02.615345 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:02.615272 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/afdcb83e-8517-4833-af04-22feff9dc221-oauth-serving-cert\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:20:02.615345 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:02.615297 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afdcb83e-8517-4833-af04-22feff9dc221-trusted-ca-bundle\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:20:02.615345 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:02.615306 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/afdcb83e-8517-4833-af04-22feff9dc221-console-oauth-config\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:20:02.615345 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:02.615314 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/afdcb83e-8517-4833-af04-22feff9dc221-service-ca\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:20:02.615345 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:02.615322 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/afdcb83e-8517-4833-af04-22feff9dc221-console-serving-cert\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:20:02.615345 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:02.615332 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/afdcb83e-8517-4833-af04-22feff9dc221-console-config\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:20:02.615345 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:02.615340 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8jnx\" (UniqueName: \"kubernetes.io/projected/afdcb83e-8517-4833-af04-22feff9dc221-kube-api-access-d8jnx\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:20:03.210605 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:03.210573 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b565596bf-z8wcz" event={"ID":"afdcb83e-8517-4833-af04-22feff9dc221","Type":"ContainerDied","Data":"e349ade2aaea18772f3681e2acc4eecb19f0fb2cfba498cefd24cf5bd23dba2c"} Sep 10 12:20:03.211040 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:03.210612 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b565596bf-z8wcz" Sep 10 12:20:03.211040 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:03.210642 2580 scope.go:117] "RemoveContainer" containerID="3741c1eb9bd3cc9f4f42301d40d04e04aa31f353f3c62f50e3b48e2808480286" Sep 10 12:20:03.234979 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:03.234956 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b565596bf-z8wcz"] Sep 10 12:20:03.243559 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:03.243539 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b565596bf-z8wcz"] Sep 10 12:20:04.556569 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:04.556537 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afdcb83e-8517-4833-af04-22feff9dc221" path="/var/lib/kubelet/pods/afdcb83e-8517-4833-af04-22feff9dc221/volumes" Sep 10 12:20:06.486249 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:06.486222 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5bbdc6f8cb-sck44" Sep 10 12:20:06.489733 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:06.489709 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5bbdc6f8cb-sck44" Sep 10 12:20:09.375314 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:09.375280 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7169f42e-6be8-4806-bf45-c77b6e2bdfd1-metrics-certs\") pod \"network-metrics-daemon-7vx2g\" (UID: \"7169f42e-6be8-4806-bf45-c77b6e2bdfd1\") " pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:20:09.378718 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:09.378696 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 12:20:09.388185 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:09.388161 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7169f42e-6be8-4806-bf45-c77b6e2bdfd1-metrics-certs\") pod \"network-metrics-daemon-7vx2g\" (UID: \"7169f42e-6be8-4806-bf45-c77b6e2bdfd1\") " pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:20:09.467598 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:09.467571 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wdbdh\"" Sep 10 12:20:09.475051 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:09.475027 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vx2g" Sep 10 12:20:09.476444 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:09.476423 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-852dl\" (UniqueName: \"kubernetes.io/projected/1b89f971-1fee-46bf-ad4c-e68e6b83b9a0-kube-api-access-852dl\") pod \"network-check-target-d7ls5\" (UID: \"1b89f971-1fee-46bf-ad4c-e68e6b83b9a0\") " pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:20:09.479388 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:09.479366 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 10 12:20:09.489420 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:09.489397 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 10 12:20:09.500299 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:09.500273 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-852dl\" (UniqueName: \"kubernetes.io/projected/1b89f971-1fee-46bf-ad4c-e68e6b83b9a0-kube-api-access-852dl\") pod \"network-check-target-d7ls5\" (UID: \"1b89f971-1fee-46bf-ad4c-e68e6b83b9a0\") " pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:20:09.753828 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:09.753794 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7vx2g"] Sep 10 12:20:09.758670 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:20:09.758643 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7169f42e_6be8_4806_bf45_c77b6e2bdfd1.slice/crio-b516613e7cee4f6c39a623543939dfbdb79555fd6b12ae662fb632aa5968897e WatchSource:0}: Error finding container b516613e7cee4f6c39a623543939dfbdb79555fd6b12ae662fb632aa5968897e: Status 404 returned error can't find the container with id b516613e7cee4f6c39a623543939dfbdb79555fd6b12ae662fb632aa5968897e Sep 10 12:20:09.774775 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:09.774742 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cfjr5\"" Sep 10 12:20:09.783020 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:09.782996 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:20:10.069849 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:10.069825 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-d7ls5"] Sep 10 12:20:10.072281 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:20:10.072252 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b89f971_1fee_46bf_ad4c_e68e6b83b9a0.slice/crio-fc3dc838b72fbbfd0e71a6b94a8ab1e8391e7e964f6b8ce074abad095338c439 WatchSource:0}: Error finding container fc3dc838b72fbbfd0e71a6b94a8ab1e8391e7e964f6b8ce074abad095338c439: Status 404 returned error can't find the container with id fc3dc838b72fbbfd0e71a6b94a8ab1e8391e7e964f6b8ce074abad095338c439 Sep 10 12:20:10.243519 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:10.243489 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-d7ls5" event={"ID":"1b89f971-1fee-46bf-ad4c-e68e6b83b9a0","Type":"ContainerStarted","Data":"fc3dc838b72fbbfd0e71a6b94a8ab1e8391e7e964f6b8ce074abad095338c439"} Sep 10 12:20:10.244631 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:10.244609 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7vx2g" event={"ID":"7169f42e-6be8-4806-bf45-c77b6e2bdfd1","Type":"ContainerStarted","Data":"b516613e7cee4f6c39a623543939dfbdb79555fd6b12ae662fb632aa5968897e"} Sep 10 12:20:11.251022 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:11.250984 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7vx2g" event={"ID":"7169f42e-6be8-4806-bf45-c77b6e2bdfd1","Type":"ContainerStarted","Data":"a7dfda64d3d9288bfb6c12829888f7be47c7002d0baf22786e845bd2558f8d17"} Sep 10 12:20:12.260996 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:12.260961 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7vx2g" event={"ID":"7169f42e-6be8-4806-bf45-c77b6e2bdfd1","Type":"ContainerStarted","Data":"831e39be4a9ddba22e08ae3334c89a1df6ed1f3135298342349a50727e6f3723"} Sep 10 12:20:12.279466 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:12.279419 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-7vx2g" podStartSLOduration=131.050291978 podStartE2EDuration="2m12.279403511s" totalImagesPullingTime="1.229111533s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:18:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:20:12.278454973 +0000 UTC m=+132.285771390" watchObservedRunningTime="2026-09-10 12:20:12.279403511 +0000 UTC m=+132.286719892" Sep 10 12:20:14.272526 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:14.272488 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-d7ls5" event={"ID":"1b89f971-1fee-46bf-ad4c-e68e6b83b9a0","Type":"ContainerStarted","Data":"4e61744b58be5169bb35688a6e8a0092f42b630d969c7400bec14c75e6fa8b4a"} Sep 10 12:20:14.272896 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:14.272570 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:20:14.291344 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:14.291303 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-d7ls5" podStartSLOduration=130.679421554 podStartE2EDuration="2m14.291288677s" totalImagesPullingTime="3.611867123s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:18:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:20:14.289982181 +0000 UTC m=+134.297298564" watchObservedRunningTime="2026-09-10 12:20:14.291288677 +0000 UTC m=+134.298605084" Sep 10 12:20:28.301961 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:28.301913 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:20:28.320629 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:28.320602 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:20:28.341429 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:28.341405 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 12:20:45.279551 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:45.279518 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-d7ls5" Sep 10 12:20:46.042419 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.042385 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-79cb7874c6-brgtb"] Sep 10 12:20:46.043123 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.043096 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afdcb83e-8517-4833-af04-22feff9dc221" containerName="console" Sep 10 12:20:46.043123 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.043122 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="afdcb83e-8517-4833-af04-22feff9dc221" containerName="console" Sep 10 12:20:46.043278 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.043202 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="afdcb83e-8517-4833-af04-22feff9dc221" containerName="console" Sep 10 12:20:46.045624 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.045609 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-79cb7874c6-brgtb" Sep 10 12:20:46.053561 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.053531 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 10 12:20:46.054078 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.054057 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 10 12:20:46.054351 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.054330 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 10 12:20:46.054351 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.054340 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 10 12:20:46.054896 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.054852 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 10 12:20:46.055692 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.055673 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-dsnsk\"" Sep 10 12:20:46.073435 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.073410 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 10 12:20:46.156545 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.156513 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-79cb7874c6-brgtb"] Sep 10 12:20:46.206781 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.206751 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ca0765fc-d4d3-43fd-acae-3f0e0b3f5971-federate-client-tls\") pod \"telemeter-client-79cb7874c6-brgtb\" (UID: \"ca0765fc-d4d3-43fd-acae-3f0e0b3f5971\") " pod="openshift-monitoring/telemeter-client-79cb7874c6-brgtb" Sep 10 12:20:46.206781 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.206780 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca0765fc-d4d3-43fd-acae-3f0e0b3f5971-telemeter-trusted-ca-bundle\") pod \"telemeter-client-79cb7874c6-brgtb\" (UID: \"ca0765fc-d4d3-43fd-acae-3f0e0b3f5971\") " pod="openshift-monitoring/telemeter-client-79cb7874c6-brgtb" Sep 10 12:20:46.206994 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.206808 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ca0765fc-d4d3-43fd-acae-3f0e0b3f5971-telemeter-client-tls\") pod \"telemeter-client-79cb7874c6-brgtb\" (UID: \"ca0765fc-d4d3-43fd-acae-3f0e0b3f5971\") " pod="openshift-monitoring/telemeter-client-79cb7874c6-brgtb" Sep 10 12:20:46.206994 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.206953 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ca0765fc-d4d3-43fd-acae-3f0e0b3f5971-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-79cb7874c6-brgtb\" (UID: \"ca0765fc-d4d3-43fd-acae-3f0e0b3f5971\") " pod="openshift-monitoring/telemeter-client-79cb7874c6-brgtb" Sep 10 12:20:46.206994 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.206984 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54r22\" (UniqueName: \"kubernetes.io/projected/ca0765fc-d4d3-43fd-acae-3f0e0b3f5971-kube-api-access-54r22\") pod \"telemeter-client-79cb7874c6-brgtb\" (UID: \"ca0765fc-d4d3-43fd-acae-3f0e0b3f5971\") " pod="openshift-monitoring/telemeter-client-79cb7874c6-brgtb" Sep 10 12:20:46.207087 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.207007 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca0765fc-d4d3-43fd-acae-3f0e0b3f5971-serving-certs-ca-bundle\") pod \"telemeter-client-79cb7874c6-brgtb\" (UID: \"ca0765fc-d4d3-43fd-acae-3f0e0b3f5971\") " pod="openshift-monitoring/telemeter-client-79cb7874c6-brgtb" Sep 10 12:20:46.207087 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.207060 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ca0765fc-d4d3-43fd-acae-3f0e0b3f5971-secret-telemeter-client\") pod \"telemeter-client-79cb7874c6-brgtb\" (UID: \"ca0765fc-d4d3-43fd-acae-3f0e0b3f5971\") " pod="openshift-monitoring/telemeter-client-79cb7874c6-brgtb" Sep 10 12:20:46.207087 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.207083 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ca0765fc-d4d3-43fd-acae-3f0e0b3f5971-metrics-client-ca\") pod \"telemeter-client-79cb7874c6-brgtb\" (UID: \"ca0765fc-d4d3-43fd-acae-3f0e0b3f5971\") " pod="openshift-monitoring/telemeter-client-79cb7874c6-brgtb" Sep 10 12:20:46.308145 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.308071 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ca0765fc-d4d3-43fd-acae-3f0e0b3f5971-federate-client-tls\") pod \"telemeter-client-79cb7874c6-brgtb\" (UID: \"ca0765fc-d4d3-43fd-acae-3f0e0b3f5971\") " pod="openshift-monitoring/telemeter-client-79cb7874c6-brgtb" Sep 10 12:20:46.308145 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.308108 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca0765fc-d4d3-43fd-acae-3f0e0b3f5971-telemeter-trusted-ca-bundle\") pod \"telemeter-client-79cb7874c6-brgtb\" (UID: \"ca0765fc-d4d3-43fd-acae-3f0e0b3f5971\") " pod="openshift-monitoring/telemeter-client-79cb7874c6-brgtb" Sep 10 12:20:46.308145 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.308133 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ca0765fc-d4d3-43fd-acae-3f0e0b3f5971-telemeter-client-tls\") pod \"telemeter-client-79cb7874c6-brgtb\" (UID: \"ca0765fc-d4d3-43fd-acae-3f0e0b3f5971\") " pod="openshift-monitoring/telemeter-client-79cb7874c6-brgtb" Sep 10 12:20:46.308669 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.308175 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ca0765fc-d4d3-43fd-acae-3f0e0b3f5971-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-79cb7874c6-brgtb\" (UID: \"ca0765fc-d4d3-43fd-acae-3f0e0b3f5971\") " pod="openshift-monitoring/telemeter-client-79cb7874c6-brgtb" Sep 10 12:20:46.308669 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.308201 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54r22\" (UniqueName: \"kubernetes.io/projected/ca0765fc-d4d3-43fd-acae-3f0e0b3f5971-kube-api-access-54r22\") pod \"telemeter-client-79cb7874c6-brgtb\" (UID: \"ca0765fc-d4d3-43fd-acae-3f0e0b3f5971\") " pod="openshift-monitoring/telemeter-client-79cb7874c6-brgtb" Sep 10 12:20:46.308669 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.308235 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca0765fc-d4d3-43fd-acae-3f0e0b3f5971-serving-certs-ca-bundle\") pod \"telemeter-client-79cb7874c6-brgtb\" (UID: \"ca0765fc-d4d3-43fd-acae-3f0e0b3f5971\") " pod="openshift-monitoring/telemeter-client-79cb7874c6-brgtb" Sep 10 12:20:46.308669 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.308378 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ca0765fc-d4d3-43fd-acae-3f0e0b3f5971-secret-telemeter-client\") pod \"telemeter-client-79cb7874c6-brgtb\" (UID: \"ca0765fc-d4d3-43fd-acae-3f0e0b3f5971\") " pod="openshift-monitoring/telemeter-client-79cb7874c6-brgtb" Sep 10 12:20:46.308669 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.308419 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ca0765fc-d4d3-43fd-acae-3f0e0b3f5971-metrics-client-ca\") pod \"telemeter-client-79cb7874c6-brgtb\" (UID: \"ca0765fc-d4d3-43fd-acae-3f0e0b3f5971\") " pod="openshift-monitoring/telemeter-client-79cb7874c6-brgtb" Sep 10 12:20:46.309045 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.309008 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca0765fc-d4d3-43fd-acae-3f0e0b3f5971-serving-certs-ca-bundle\") pod \"telemeter-client-79cb7874c6-brgtb\" (UID: \"ca0765fc-d4d3-43fd-acae-3f0e0b3f5971\") " pod="openshift-monitoring/telemeter-client-79cb7874c6-brgtb" Sep 10 12:20:46.309114 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.309075 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca0765fc-d4d3-43fd-acae-3f0e0b3f5971-telemeter-trusted-ca-bundle\") pod \"telemeter-client-79cb7874c6-brgtb\" (UID: \"ca0765fc-d4d3-43fd-acae-3f0e0b3f5971\") " pod="openshift-monitoring/telemeter-client-79cb7874c6-brgtb" Sep 10 12:20:46.309174 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.309153 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ca0765fc-d4d3-43fd-acae-3f0e0b3f5971-metrics-client-ca\") pod \"telemeter-client-79cb7874c6-brgtb\" (UID: \"ca0765fc-d4d3-43fd-acae-3f0e0b3f5971\") " pod="openshift-monitoring/telemeter-client-79cb7874c6-brgtb" Sep 10 12:20:46.310750 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.310724 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ca0765fc-d4d3-43fd-acae-3f0e0b3f5971-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-79cb7874c6-brgtb\" (UID: \"ca0765fc-d4d3-43fd-acae-3f0e0b3f5971\") " pod="openshift-monitoring/telemeter-client-79cb7874c6-brgtb" Sep 10 12:20:46.310894 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.310733 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ca0765fc-d4d3-43fd-acae-3f0e0b3f5971-federate-client-tls\") pod \"telemeter-client-79cb7874c6-brgtb\" (UID: \"ca0765fc-d4d3-43fd-acae-3f0e0b3f5971\") " pod="openshift-monitoring/telemeter-client-79cb7874c6-brgtb" Sep 10 12:20:46.310967 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.310936 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ca0765fc-d4d3-43fd-acae-3f0e0b3f5971-telemeter-client-tls\") pod \"telemeter-client-79cb7874c6-brgtb\" (UID: \"ca0765fc-d4d3-43fd-acae-3f0e0b3f5971\") " pod="openshift-monitoring/telemeter-client-79cb7874c6-brgtb" Sep 10 12:20:46.310967 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.310948 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ca0765fc-d4d3-43fd-acae-3f0e0b3f5971-secret-telemeter-client\") pod \"telemeter-client-79cb7874c6-brgtb\" (UID: \"ca0765fc-d4d3-43fd-acae-3f0e0b3f5971\") " pod="openshift-monitoring/telemeter-client-79cb7874c6-brgtb" Sep 10 12:20:46.317240 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.317218 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54r22\" (UniqueName: \"kubernetes.io/projected/ca0765fc-d4d3-43fd-acae-3f0e0b3f5971-kube-api-access-54r22\") pod \"telemeter-client-79cb7874c6-brgtb\" (UID: \"ca0765fc-d4d3-43fd-acae-3f0e0b3f5971\") " pod="openshift-monitoring/telemeter-client-79cb7874c6-brgtb" Sep 10 12:20:46.357330 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.357312 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-79cb7874c6-brgtb" Sep 10 12:20:46.639372 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:46.639344 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-79cb7874c6-brgtb"] Sep 10 12:20:46.640372 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:20:46.640343 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca0765fc_d4d3_43fd_acae_3f0e0b3f5971.slice/crio-676aac7bac665c5408725286e3de6c46530993a1b0b12d0cc575ac0cc3640ce4 WatchSource:0}: Error finding container 676aac7bac665c5408725286e3de6c46530993a1b0b12d0cc575ac0cc3640ce4: Status 404 returned error can't find the container with id 676aac7bac665c5408725286e3de6c46530993a1b0b12d0cc575ac0cc3640ce4 Sep 10 12:20:47.403109 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:47.403079 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-79cb7874c6-brgtb" event={"ID":"ca0765fc-d4d3-43fd-acae-3f0e0b3f5971","Type":"ContainerStarted","Data":"676aac7bac665c5408725286e3de6c46530993a1b0b12d0cc575ac0cc3640ce4"} Sep 10 12:20:49.413556 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:49.413526 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-79cb7874c6-brgtb" event={"ID":"ca0765fc-d4d3-43fd-acae-3f0e0b3f5971","Type":"ContainerStarted","Data":"fc829969c2649d9e6ac784054458984e537f7f520146a2609aac80cf1cb57bb8"} Sep 10 12:20:49.413928 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:49.413609 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-79cb7874c6-brgtb" event={"ID":"ca0765fc-d4d3-43fd-acae-3f0e0b3f5971","Type":"ContainerStarted","Data":"02dff24dfc9326cd937b38086833d749a6c4827bfb6b891b92a25f3fd13d570d"} Sep 10 12:20:49.413928 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:49.413629 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-79cb7874c6-brgtb" event={"ID":"ca0765fc-d4d3-43fd-acae-3f0e0b3f5971","Type":"ContainerStarted","Data":"494ca9e711bfe00d17b4e7d6f3bb68009d0b24eb6cebc2193104226bf74187d5"} Sep 10 12:20:49.441284 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:20:49.441234 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-79cb7874c6-brgtb" podStartSLOduration=1.4567277 podStartE2EDuration="3.441219697s" totalImagesPullingTime="1.984491997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:20:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:20:49.439365224 +0000 UTC m=+169.446681605" watchObservedRunningTime="2026-09-10 12:20:49.441219697 +0000 UTC m=+169.448536078" Sep 10 12:23:00.414131 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:00.414103 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tfcmz_e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58/ovn-acl-logging/0.log" Sep 10 12:23:00.414766 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:00.414745 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tfcmz_e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58/ovn-acl-logging/0.log" Sep 10 12:23:00.422198 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:00.422177 2580 kubelet.go:1648] "Image garbage collection succeeded" Sep 10 12:23:38.747481 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:38.747447 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm"] Sep 10 12:23:38.751000 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:38.750984 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:23:38.754642 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:38.754619 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\"" Sep 10 12:23:38.754836 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:38.754817 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 10 12:23:38.754960 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:38.754824 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 10 12:23:38.754960 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:38.754912 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 10 12:23:38.754960 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:38.754938 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 10 12:23:38.755125 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:38.755067 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 10 12:23:38.755598 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:38.755576 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 10 12:23:38.755598 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:38.755581 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-g2mp2\"" Sep 10 12:23:38.768536 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:38.768511 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm"] Sep 10 12:23:38.891916 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:38.891881 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-ui-tls\") pod \"feast-simple-feast-setup-67d6b6f8f7-nvxjm\" (UID: \"5a7f8f08-a9fc-4302-8495-f1a2f678d28e\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:23:38.892085 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:38.891995 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-feast-data\") pod \"feast-simple-feast-setup-67d6b6f8f7-nvxjm\" (UID: \"5a7f8f08-a9fc-4302-8495-f1a2f678d28e\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:23:38.892085 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:38.892038 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-registry-tls\") pod \"feast-simple-feast-setup-67d6b6f8f7-nvxjm\" (UID: \"5a7f8f08-a9fc-4302-8495-f1a2f678d28e\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:23:38.892085 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:38.892067 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-online-tls\") pod \"feast-simple-feast-setup-67d6b6f8f7-nvxjm\" (UID: \"5a7f8f08-a9fc-4302-8495-f1a2f678d28e\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:23:38.892196 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:38.892085 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gz6b6\" (UniqueName: \"kubernetes.io/projected/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-kube-api-access-gz6b6\") pod \"feast-simple-feast-setup-67d6b6f8f7-nvxjm\" (UID: \"5a7f8f08-a9fc-4302-8495-f1a2f678d28e\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:23:38.892196 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:38.892121 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-offline-tls\") pod \"feast-simple-feast-setup-67d6b6f8f7-nvxjm\" (UID: \"5a7f8f08-a9fc-4302-8495-f1a2f678d28e\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:23:38.992941 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:38.992903 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-feast-data\") pod \"feast-simple-feast-setup-67d6b6f8f7-nvxjm\" (UID: \"5a7f8f08-a9fc-4302-8495-f1a2f678d28e\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:23:38.992941 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:38.992943 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-registry-tls\") pod \"feast-simple-feast-setup-67d6b6f8f7-nvxjm\" (UID: \"5a7f8f08-a9fc-4302-8495-f1a2f678d28e\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:23:38.993165 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:38.992969 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-online-tls\") pod \"feast-simple-feast-setup-67d6b6f8f7-nvxjm\" (UID: \"5a7f8f08-a9fc-4302-8495-f1a2f678d28e\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:23:38.993165 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:38.992997 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gz6b6\" (UniqueName: \"kubernetes.io/projected/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-kube-api-access-gz6b6\") pod \"feast-simple-feast-setup-67d6b6f8f7-nvxjm\" (UID: \"5a7f8f08-a9fc-4302-8495-f1a2f678d28e\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:23:38.993165 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:38.993023 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-offline-tls\") pod \"feast-simple-feast-setup-67d6b6f8f7-nvxjm\" (UID: \"5a7f8f08-a9fc-4302-8495-f1a2f678d28e\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:23:38.993165 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:38.993069 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-ui-tls\") pod \"feast-simple-feast-setup-67d6b6f8f7-nvxjm\" (UID: \"5a7f8f08-a9fc-4302-8495-f1a2f678d28e\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:23:38.993165 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:23:38.993078 2580 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Sep 10 12:23:38.993165 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:23:38.993145 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-registry-tls podName:5a7f8f08-a9fc-4302-8495-f1a2f678d28e nodeName:}" failed. No retries permitted until 2026-09-10 12:23:39.493123188 +0000 UTC m=+339.500439564 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-registry-tls") pod "feast-simple-feast-setup-67d6b6f8f7-nvxjm" (UID: "5a7f8f08-a9fc-4302-8495-f1a2f678d28e") : secret "feast-simple-feast-setup-registry-tls" not found Sep 10 12:23:38.993399 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:23:38.993166 2580 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Sep 10 12:23:38.993399 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:23:38.993182 2580 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Sep 10 12:23:38.993399 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:23:38.993230 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-offline-tls podName:5a7f8f08-a9fc-4302-8495-f1a2f678d28e nodeName:}" failed. No retries permitted until 2026-09-10 12:23:39.493212121 +0000 UTC m=+339.500528487 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-offline-tls") pod "feast-simple-feast-setup-67d6b6f8f7-nvxjm" (UID: "5a7f8f08-a9fc-4302-8495-f1a2f678d28e") : secret "feast-simple-feast-setup-offline-tls" not found Sep 10 12:23:38.993399 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:23:38.993249 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-ui-tls podName:5a7f8f08-a9fc-4302-8495-f1a2f678d28e nodeName:}" failed. No retries permitted until 2026-09-10 12:23:39.493240399 +0000 UTC m=+339.500556760 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-ui-tls") pod "feast-simple-feast-setup-67d6b6f8f7-nvxjm" (UID: "5a7f8f08-a9fc-4302-8495-f1a2f678d28e") : secret "feast-simple-feast-setup-ui-tls" not found Sep 10 12:23:38.993399 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:38.993325 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-feast-data\") pod \"feast-simple-feast-setup-67d6b6f8f7-nvxjm\" (UID: \"5a7f8f08-a9fc-4302-8495-f1a2f678d28e\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:23:38.995320 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:38.995300 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-online-tls\") pod \"feast-simple-feast-setup-67d6b6f8f7-nvxjm\" (UID: \"5a7f8f08-a9fc-4302-8495-f1a2f678d28e\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:23:39.002611 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:39.002543 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gz6b6\" (UniqueName: \"kubernetes.io/projected/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-kube-api-access-gz6b6\") pod \"feast-simple-feast-setup-67d6b6f8f7-nvxjm\" (UID: \"5a7f8f08-a9fc-4302-8495-f1a2f678d28e\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:23:39.497057 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:39.497027 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-ui-tls\") pod \"feast-simple-feast-setup-67d6b6f8f7-nvxjm\" (UID: \"5a7f8f08-a9fc-4302-8495-f1a2f678d28e\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:23:39.497233 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:39.497120 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-registry-tls\") pod \"feast-simple-feast-setup-67d6b6f8f7-nvxjm\" (UID: \"5a7f8f08-a9fc-4302-8495-f1a2f678d28e\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:23:39.497233 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:39.497169 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-offline-tls\") pod \"feast-simple-feast-setup-67d6b6f8f7-nvxjm\" (UID: \"5a7f8f08-a9fc-4302-8495-f1a2f678d28e\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:23:39.499621 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:39.499594 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-offline-tls\") pod \"feast-simple-feast-setup-67d6b6f8f7-nvxjm\" (UID: \"5a7f8f08-a9fc-4302-8495-f1a2f678d28e\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:23:39.499621 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:39.499617 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-ui-tls\") pod \"feast-simple-feast-setup-67d6b6f8f7-nvxjm\" (UID: \"5a7f8f08-a9fc-4302-8495-f1a2f678d28e\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:23:39.499767 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:39.499648 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-registry-tls\") pod \"feast-simple-feast-setup-67d6b6f8f7-nvxjm\" (UID: \"5a7f8f08-a9fc-4302-8495-f1a2f678d28e\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:23:39.668726 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:39.668690 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:23:39.953384 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:39.953361 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm"] Sep 10 12:23:39.955380 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:23:39.955353 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5a7f8f08_a9fc_4302_8495_f1a2f678d28e.slice/crio-12d0cb5b85c6a2de582caf56223bf803444edbdbf72e9eda58d70fd5f2e73902 WatchSource:0}: Error finding container 12d0cb5b85c6a2de582caf56223bf803444edbdbf72e9eda58d70fd5f2e73902: Status 404 returned error can't find the container with id 12d0cb5b85c6a2de582caf56223bf803444edbdbf72e9eda58d70fd5f2e73902 Sep 10 12:23:39.957389 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:39.957374 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 12:23:40.096368 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:23:40.096341 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" event={"ID":"5a7f8f08-a9fc-4302-8495-f1a2f678d28e","Type":"ContainerStarted","Data":"12d0cb5b85c6a2de582caf56223bf803444edbdbf72e9eda58d70fd5f2e73902"} Sep 10 12:24:08.224109 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:24:08.224070 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" event={"ID":"5a7f8f08-a9fc-4302-8495-f1a2f678d28e","Type":"ContainerStarted","Data":"98b5d7036514d0d153f3bd5d340b21aafbcff976a7b3195aaf0c8953ec45dd1d"} Sep 10 12:24:13.246248 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:24:13.246210 2580 generic.go:356] "Generic (PLEG): container finished" podID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerID="98b5d7036514d0d153f3bd5d340b21aafbcff976a7b3195aaf0c8953ec45dd1d" exitCode=0 Sep 10 12:24:13.246768 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:24:13.246284 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" event={"ID":"5a7f8f08-a9fc-4302-8495-f1a2f678d28e","Type":"ContainerDied","Data":"98b5d7036514d0d153f3bd5d340b21aafbcff976a7b3195aaf0c8953ec45dd1d"} Sep 10 12:24:14.253276 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:24:14.253244 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" event={"ID":"5a7f8f08-a9fc-4302-8495-f1a2f678d28e","Type":"ContainerStarted","Data":"30ad9c82c8aef0aebc90c2e54fd29df6b007711403e9b8c4bdde9225ba1dae5a"} Sep 10 12:24:19.282716 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:24:19.282683 2580 generic.go:356] "Generic (PLEG): container finished" podID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerID="30ad9c82c8aef0aebc90c2e54fd29df6b007711403e9b8c4bdde9225ba1dae5a" exitCode=0 Sep 10 12:24:19.283088 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:24:19.282756 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" event={"ID":"5a7f8f08-a9fc-4302-8495-f1a2f678d28e","Type":"ContainerDied","Data":"30ad9c82c8aef0aebc90c2e54fd29df6b007711403e9b8c4bdde9225ba1dae5a"} Sep 10 12:24:19.293124 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:24:19.293106 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\"" Sep 10 12:24:20.293595 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:24:20.293549 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" event={"ID":"5a7f8f08-a9fc-4302-8495-f1a2f678d28e","Type":"ContainerStarted","Data":"f7cc74d160f479f95a826718ef8f8e35e67c879eb416c5d705d46383c47b60b4"} Sep 10 12:24:20.294086 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:24:20.293613 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" event={"ID":"5a7f8f08-a9fc-4302-8495-f1a2f678d28e","Type":"ContainerStarted","Data":"1987a05dd027f4e55de68e4bd61572c31a1d6c2a2e8cae26d151793f8c25aa7c"} Sep 10 12:24:20.294086 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:24:20.293636 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" event={"ID":"5a7f8f08-a9fc-4302-8495-f1a2f678d28e","Type":"ContainerStarted","Data":"9dda20e8302d2c6c5acfbd110ab8819f0d78f7b279438a626aae92b99069e836"} Sep 10 12:24:20.294086 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:24:20.293656 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" event={"ID":"5a7f8f08-a9fc-4302-8495-f1a2f678d28e","Type":"ContainerStarted","Data":"058063eba96edeec74c71aa0d93c3ba404c5a235a2f4938d8892d27d1ad56039"} Sep 10 12:24:20.325538 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:24:20.325477 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" podStartSLOduration=14.564469407 podStartE2EDuration="42.325454543s" totalImagesPullingTime="27.760985136s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:23:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:24:20.322161108 +0000 UTC m=+380.329477528" watchObservedRunningTime="2026-09-10 12:24:20.325454543 +0000 UTC m=+380.332770927" Sep 10 12:24:21.669321 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:24:21.669247 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:24:21.669321 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:24:21.669308 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:24:21.669321 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:24:21.669332 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:24:21.669906 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:24:21.669759 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:24:21.671629 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:24:21.671553 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" podUID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.20:8443: connect: connection refused" Sep 10 12:24:21.671629 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:24:21.671591 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" podUID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.20:8816: connect: connection refused" Sep 10 12:24:21.671629 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:24:21.671618 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" podUID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.20:6571: connect: connection refused" Sep 10 12:24:21.671892 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:24:21.671597 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" podUID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.20:6567/health\": dial tcp 10.134.0.20:6567: connect: connection refused" Sep 10 12:24:24.669445 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:24:24.669399 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" podUID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.20:6571: connect: connection refused" Sep 10 12:24:24.670047 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:24:24.669418 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" podUID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.20:8816: connect: connection refused" Sep 10 12:24:24.670171 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:24:24.669407 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" podUID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.20:8443: connect: connection refused" Sep 10 12:24:24.670171 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:24:24.669510 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" podUID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.20:6567/health\": dial tcp 10.134.0.20:6567: connect: connection refused" Sep 10 12:24:27.669442 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:24:27.669375 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" podUID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.20:8443: connect: connection refused" Sep 10 12:24:27.670631 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:24:27.670591 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:24:27.670953 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:24:27.670750 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:24:27.671360 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:24:27.671328 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:24:27.671499 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:24:27.671380 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:24:27.671944 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:24:27.671909 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:24:27.672114 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:24:27.672051 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:24:27.677487 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:24:27.677446 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:24:28.342901 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:24:28.342828 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:24:28.346320 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:24:28.346295 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:24:29.669211 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:24:29.669166 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:24:30.670060 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:24:30.670028 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:24:31.356430 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:24:31.356389 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:25:33.054728 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:33.054636 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm"] Sep 10 12:25:33.055320 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:33.054994 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" podUID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerName="registry" containerID="cri-o://058063eba96edeec74c71aa0d93c3ba404c5a235a2f4938d8892d27d1ad56039" gracePeriod=30 Sep 10 12:25:33.055320 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:33.055042 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" podUID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerName="online" containerID="cri-o://9dda20e8302d2c6c5acfbd110ab8819f0d78f7b279438a626aae92b99069e836" gracePeriod=30 Sep 10 12:25:33.055320 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:33.055054 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" podUID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerName="offline" containerID="cri-o://1987a05dd027f4e55de68e4bd61572c31a1d6c2a2e8cae26d151793f8c25aa7c" gracePeriod=30 Sep 10 12:25:33.055320 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:33.055054 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" podUID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerName="ui" containerID="cri-o://f7cc74d160f479f95a826718ef8f8e35e67c879eb416c5d705d46383c47b60b4" gracePeriod=30 Sep 10 12:25:33.446771 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:33.446731 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9"] Sep 10 12:25:33.450908 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:33.450880 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:25:33.453748 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:33.453722 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-lf76n\"" Sep 10 12:25:33.468727 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:33.468682 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9"] Sep 10 12:25:33.540640 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:33.540591 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0570e6d2-af94-447b-a8e2-73b63855d2ee-ui-tls\") pod \"feast-simple-feast-setup-67d6b6f8f7-b2sx9\" (UID: \"0570e6d2-af94-447b-a8e2-73b63855d2ee\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:25:33.540640 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:33.540646 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqv7h\" (UniqueName: \"kubernetes.io/projected/0570e6d2-af94-447b-a8e2-73b63855d2ee-kube-api-access-mqv7h\") pod \"feast-simple-feast-setup-67d6b6f8f7-b2sx9\" (UID: \"0570e6d2-af94-447b-a8e2-73b63855d2ee\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:25:33.540943 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:33.540712 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0570e6d2-af94-447b-a8e2-73b63855d2ee-feast-data\") pod \"feast-simple-feast-setup-67d6b6f8f7-b2sx9\" (UID: \"0570e6d2-af94-447b-a8e2-73b63855d2ee\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:25:33.540943 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:33.540789 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0570e6d2-af94-447b-a8e2-73b63855d2ee-online-tls\") pod \"feast-simple-feast-setup-67d6b6f8f7-b2sx9\" (UID: \"0570e6d2-af94-447b-a8e2-73b63855d2ee\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:25:33.540943 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:33.540886 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0570e6d2-af94-447b-a8e2-73b63855d2ee-registry-tls\") pod \"feast-simple-feast-setup-67d6b6f8f7-b2sx9\" (UID: \"0570e6d2-af94-447b-a8e2-73b63855d2ee\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:25:33.540943 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:33.540917 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0570e6d2-af94-447b-a8e2-73b63855d2ee-offline-tls\") pod \"feast-simple-feast-setup-67d6b6f8f7-b2sx9\" (UID: \"0570e6d2-af94-447b-a8e2-73b63855d2ee\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:25:33.641760 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:33.641709 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0570e6d2-af94-447b-a8e2-73b63855d2ee-ui-tls\") pod \"feast-simple-feast-setup-67d6b6f8f7-b2sx9\" (UID: \"0570e6d2-af94-447b-a8e2-73b63855d2ee\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:25:33.641760 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:33.641760 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqv7h\" (UniqueName: \"kubernetes.io/projected/0570e6d2-af94-447b-a8e2-73b63855d2ee-kube-api-access-mqv7h\") pod \"feast-simple-feast-setup-67d6b6f8f7-b2sx9\" (UID: \"0570e6d2-af94-447b-a8e2-73b63855d2ee\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:25:33.642025 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:33.641795 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0570e6d2-af94-447b-a8e2-73b63855d2ee-feast-data\") pod \"feast-simple-feast-setup-67d6b6f8f7-b2sx9\" (UID: \"0570e6d2-af94-447b-a8e2-73b63855d2ee\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:25:33.642025 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:33.641837 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0570e6d2-af94-447b-a8e2-73b63855d2ee-online-tls\") pod \"feast-simple-feast-setup-67d6b6f8f7-b2sx9\" (UID: \"0570e6d2-af94-447b-a8e2-73b63855d2ee\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:25:33.642025 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:25:33.641887 2580 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Sep 10 12:25:33.642025 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:33.641907 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0570e6d2-af94-447b-a8e2-73b63855d2ee-registry-tls\") pod \"feast-simple-feast-setup-67d6b6f8f7-b2sx9\" (UID: \"0570e6d2-af94-447b-a8e2-73b63855d2ee\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:25:33.642025 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:33.641929 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0570e6d2-af94-447b-a8e2-73b63855d2ee-offline-tls\") pod \"feast-simple-feast-setup-67d6b6f8f7-b2sx9\" (UID: \"0570e6d2-af94-447b-a8e2-73b63855d2ee\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:25:33.642025 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:25:33.641961 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0570e6d2-af94-447b-a8e2-73b63855d2ee-ui-tls podName:0570e6d2-af94-447b-a8e2-73b63855d2ee nodeName:}" failed. No retries permitted until 2026-09-10 12:25:34.141939503 +0000 UTC m=+454.149255864 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/0570e6d2-af94-447b-a8e2-73b63855d2ee-ui-tls") pod "feast-simple-feast-setup-67d6b6f8f7-b2sx9" (UID: "0570e6d2-af94-447b-a8e2-73b63855d2ee") : secret "feast-simple-feast-setup-ui-tls" not found Sep 10 12:25:33.642347 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:33.642280 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0570e6d2-af94-447b-a8e2-73b63855d2ee-feast-data\") pod \"feast-simple-feast-setup-67d6b6f8f7-b2sx9\" (UID: \"0570e6d2-af94-447b-a8e2-73b63855d2ee\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:25:33.644600 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:33.644565 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0570e6d2-af94-447b-a8e2-73b63855d2ee-online-tls\") pod \"feast-simple-feast-setup-67d6b6f8f7-b2sx9\" (UID: \"0570e6d2-af94-447b-a8e2-73b63855d2ee\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:25:33.645024 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:33.645001 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0570e6d2-af94-447b-a8e2-73b63855d2ee-registry-tls\") pod \"feast-simple-feast-setup-67d6b6f8f7-b2sx9\" (UID: \"0570e6d2-af94-447b-a8e2-73b63855d2ee\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:25:33.645122 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:33.645048 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0570e6d2-af94-447b-a8e2-73b63855d2ee-offline-tls\") pod \"feast-simple-feast-setup-67d6b6f8f7-b2sx9\" (UID: \"0570e6d2-af94-447b-a8e2-73b63855d2ee\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:25:33.651591 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:33.651560 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqv7h\" (UniqueName: \"kubernetes.io/projected/0570e6d2-af94-447b-a8e2-73b63855d2ee-kube-api-access-mqv7h\") pod \"feast-simple-feast-setup-67d6b6f8f7-b2sx9\" (UID: \"0570e6d2-af94-447b-a8e2-73b63855d2ee\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:25:34.146529 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:34.146490 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0570e6d2-af94-447b-a8e2-73b63855d2ee-ui-tls\") pod \"feast-simple-feast-setup-67d6b6f8f7-b2sx9\" (UID: \"0570e6d2-af94-447b-a8e2-73b63855d2ee\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:25:34.149099 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:34.149074 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0570e6d2-af94-447b-a8e2-73b63855d2ee-ui-tls\") pod \"feast-simple-feast-setup-67d6b6f8f7-b2sx9\" (UID: \"0570e6d2-af94-447b-a8e2-73b63855d2ee\") " pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:25:34.366686 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:34.366597 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:25:34.633374 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:34.633286 2580 generic.go:356] "Generic (PLEG): container finished" podID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerID="f7cc74d160f479f95a826718ef8f8e35e67c879eb416c5d705d46383c47b60b4" exitCode=0 Sep 10 12:25:34.633374 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:34.633315 2580 generic.go:356] "Generic (PLEG): container finished" podID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerID="9dda20e8302d2c6c5acfbd110ab8819f0d78f7b279438a626aae92b99069e836" exitCode=0 Sep 10 12:25:34.633374 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:34.633358 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" event={"ID":"5a7f8f08-a9fc-4302-8495-f1a2f678d28e","Type":"ContainerDied","Data":"f7cc74d160f479f95a826718ef8f8e35e67c879eb416c5d705d46383c47b60b4"} Sep 10 12:25:34.633595 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:34.633393 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" event={"ID":"5a7f8f08-a9fc-4302-8495-f1a2f678d28e","Type":"ContainerDied","Data":"9dda20e8302d2c6c5acfbd110ab8819f0d78f7b279438a626aae92b99069e836"} Sep 10 12:25:34.651000 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:34.650971 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9"] Sep 10 12:25:34.652072 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:25:34.652040 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0570e6d2_af94_447b_a8e2_73b63855d2ee.slice/crio-4b8469ea65f590e1d9b8c65bd1d81fe55fee307f6937c02d66118a45e1f0c253 WatchSource:0}: Error finding container 4b8469ea65f590e1d9b8c65bd1d81fe55fee307f6937c02d66118a45e1f0c253: Status 404 returned error can't find the container with id 4b8469ea65f590e1d9b8c65bd1d81fe55fee307f6937c02d66118a45e1f0c253 Sep 10 12:25:35.639503 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:35.639460 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" event={"ID":"0570e6d2-af94-447b-a8e2-73b63855d2ee","Type":"ContainerStarted","Data":"5397a6886e74ce86939c9b7646517157514fe4a771766be23c94837b495e7be9"} Sep 10 12:25:35.639503 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:35.639507 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" event={"ID":"0570e6d2-af94-447b-a8e2-73b63855d2ee","Type":"ContainerStarted","Data":"4b8469ea65f590e1d9b8c65bd1d81fe55fee307f6937c02d66118a45e1f0c253"} Sep 10 12:25:38.343211 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:38.343178 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" podUID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.20:6567/health\": dial tcp 10.134.0.20:6567: connect: connection refused" Sep 10 12:25:39.656811 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:39.656776 2580 generic.go:356] "Generic (PLEG): container finished" podID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerID="5397a6886e74ce86939c9b7646517157514fe4a771766be23c94837b495e7be9" exitCode=0 Sep 10 12:25:39.657274 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:39.656868 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" event={"ID":"0570e6d2-af94-447b-a8e2-73b63855d2ee","Type":"ContainerDied","Data":"5397a6886e74ce86939c9b7646517157514fe4a771766be23c94837b495e7be9"} Sep 10 12:25:40.662503 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:40.662470 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" event={"ID":"0570e6d2-af94-447b-a8e2-73b63855d2ee","Type":"ContainerStarted","Data":"1759c913c67cccf2075f0c07ec5a82c3c4a2975231626df0e01c0c7e59766506"} Sep 10 12:25:41.356138 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:41.356100 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" podUID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.20:8443: connect: connection refused" Sep 10 12:25:45.686729 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:45.686696 2580 generic.go:356] "Generic (PLEG): container finished" podID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerID="1759c913c67cccf2075f0c07ec5a82c3c4a2975231626df0e01c0c7e59766506" exitCode=0 Sep 10 12:25:45.687128 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:45.686765 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" event={"ID":"0570e6d2-af94-447b-a8e2-73b63855d2ee","Type":"ContainerDied","Data":"1759c913c67cccf2075f0c07ec5a82c3c4a2975231626df0e01c0c7e59766506"} Sep 10 12:25:46.697665 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:46.697622 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" event={"ID":"0570e6d2-af94-447b-a8e2-73b63855d2ee","Type":"ContainerStarted","Data":"dc87ea858365977bfcf8ea10957a368e05e2fe969dfbadf57aa8b3ef8c395d76"} Sep 10 12:25:46.698159 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:46.697690 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" event={"ID":"0570e6d2-af94-447b-a8e2-73b63855d2ee","Type":"ContainerStarted","Data":"f2a1c93943977c8ab16d32cc23ebeae29b0ba06520b1bf97f4ad16b10793c5f3"} Sep 10 12:25:46.698159 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:46.697712 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" event={"ID":"0570e6d2-af94-447b-a8e2-73b63855d2ee","Type":"ContainerStarted","Data":"be86f7ec3ce31061678962a95c92729fede788683b97b8e0fdfece51d6be0676"} Sep 10 12:25:46.698159 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:46.697731 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" event={"ID":"0570e6d2-af94-447b-a8e2-73b63855d2ee","Type":"ContainerStarted","Data":"dc781d101bc450900821fc6902d3e255222ece7253b0ab8eb37d413d4428729b"} Sep 10 12:25:46.724813 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:46.724741 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" podStartSLOduration=13.724718056 podStartE2EDuration="13.724718056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:25:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:25:46.722149702 +0000 UTC m=+466.729466111" watchObservedRunningTime="2026-09-10 12:25:46.724718056 +0000 UTC m=+466.732034450" Sep 10 12:25:48.343990 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:48.343929 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" podUID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.20:6567/health\": dial tcp 10.134.0.20:6567: connect: connection refused" Sep 10 12:25:49.367946 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:49.367556 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:25:49.367946 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:49.367657 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:25:49.367946 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:49.367680 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:25:49.368499 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:49.368143 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:25:49.369959 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:49.369917 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" podUID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.21:6571: connect: connection refused" Sep 10 12:25:49.370211 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:49.369937 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" podUID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.21:8443: connect: connection refused" Sep 10 12:25:49.370342 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:49.369992 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" podUID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerName="online" probeResult="failure" output="Get \"https://10.134.0.21:6567/health\": dial tcp 10.134.0.21:6567: connect: connection refused" Sep 10 12:25:49.370342 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:49.370043 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" podUID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.21:8816: connect: connection refused" Sep 10 12:25:51.356622 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:51.356577 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" podUID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.20:8443: connect: connection refused" Sep 10 12:25:52.367985 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:52.367940 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" podUID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.21:8443: connect: connection refused" Sep 10 12:25:52.368550 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:52.367939 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" podUID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.21:6571: connect: connection refused" Sep 10 12:25:52.368695 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:52.367964 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" podUID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.21:8816: connect: connection refused" Sep 10 12:25:52.368695 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:52.368029 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" podUID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerName="online" probeResult="failure" output="Get \"https://10.134.0.21:6567/health\": dial tcp 10.134.0.21:6567: connect: connection refused" Sep 10 12:25:54.367555 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:54.367519 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:25:54.367555 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:54.367566 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:25:54.368002 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:54.367587 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:25:54.368002 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:54.367602 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:25:55.368161 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:55.368119 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:25:55.368557 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:55.368306 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:25:55.368557 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:55.368422 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:25:55.369264 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:55.369238 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:25:55.369397 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:55.369295 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:25:55.369397 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:55.369326 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:25:55.373752 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:55.373731 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:25:55.756242 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:55.756215 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:25:58.342972 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:58.342929 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" podUID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.20:6567/health\": dial tcp 10.134.0.20:6567: connect: connection refused" Sep 10 12:25:58.343403 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:25:58.343074 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:26:01.355948 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:01.355909 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" podUID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.20:8443: connect: connection refused" Sep 10 12:26:01.356431 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:01.356063 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:26:03.702958 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.702931 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:26:03.793667 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.791541 2580 generic.go:356] "Generic (PLEG): container finished" podID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerID="1987a05dd027f4e55de68e4bd61572c31a1d6c2a2e8cae26d151793f8c25aa7c" exitCode=137 Sep 10 12:26:03.793667 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.791620 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" event={"ID":"5a7f8f08-a9fc-4302-8495-f1a2f678d28e","Type":"ContainerDied","Data":"1987a05dd027f4e55de68e4bd61572c31a1d6c2a2e8cae26d151793f8c25aa7c"} Sep 10 12:26:03.793667 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.791691 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" event={"ID":"5a7f8f08-a9fc-4302-8495-f1a2f678d28e","Type":"ContainerDied","Data":"058063eba96edeec74c71aa0d93c3ba404c5a235a2f4938d8892d27d1ad56039"} Sep 10 12:26:03.793667 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.791746 2580 scope.go:117] "RemoveContainer" containerID="f7cc74d160f479f95a826718ef8f8e35e67c879eb416c5d705d46383c47b60b4" Sep 10 12:26:03.793667 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.792066 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" Sep 10 12:26:03.793667 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.792209 2580 generic.go:356] "Generic (PLEG): container finished" podID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerID="058063eba96edeec74c71aa0d93c3ba404c5a235a2f4938d8892d27d1ad56039" exitCode=137 Sep 10 12:26:03.793667 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.792339 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm" event={"ID":"5a7f8f08-a9fc-4302-8495-f1a2f678d28e","Type":"ContainerDied","Data":"12d0cb5b85c6a2de582caf56223bf803444edbdbf72e9eda58d70fd5f2e73902"} Sep 10 12:26:03.805446 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.805422 2580 scope.go:117] "RemoveContainer" containerID="1987a05dd027f4e55de68e4bd61572c31a1d6c2a2e8cae26d151793f8c25aa7c" Sep 10 12:26:03.813465 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.813445 2580 scope.go:117] "RemoveContainer" containerID="9dda20e8302d2c6c5acfbd110ab8819f0d78f7b279438a626aae92b99069e836" Sep 10 12:26:03.820743 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.820726 2580 scope.go:117] "RemoveContainer" containerID="058063eba96edeec74c71aa0d93c3ba404c5a235a2f4938d8892d27d1ad56039" Sep 10 12:26:03.823950 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.823930 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-ui-tls\") pod \"5a7f8f08-a9fc-4302-8495-f1a2f678d28e\" (UID: \"5a7f8f08-a9fc-4302-8495-f1a2f678d28e\") " Sep 10 12:26:03.824042 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.823979 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-registry-tls\") pod \"5a7f8f08-a9fc-4302-8495-f1a2f678d28e\" (UID: \"5a7f8f08-a9fc-4302-8495-f1a2f678d28e\") " Sep 10 12:26:03.824162 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.824139 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-offline-tls\") pod \"5a7f8f08-a9fc-4302-8495-f1a2f678d28e\" (UID: \"5a7f8f08-a9fc-4302-8495-f1a2f678d28e\") " Sep 10 12:26:03.824229 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.824196 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gz6b6\" (UniqueName: \"kubernetes.io/projected/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-kube-api-access-gz6b6\") pod \"5a7f8f08-a9fc-4302-8495-f1a2f678d28e\" (UID: \"5a7f8f08-a9fc-4302-8495-f1a2f678d28e\") " Sep 10 12:26:03.824279 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.824265 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-feast-data\") pod \"5a7f8f08-a9fc-4302-8495-f1a2f678d28e\" (UID: \"5a7f8f08-a9fc-4302-8495-f1a2f678d28e\") " Sep 10 12:26:03.824347 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.824321 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-online-tls\") pod \"5a7f8f08-a9fc-4302-8495-f1a2f678d28e\" (UID: \"5a7f8f08-a9fc-4302-8495-f1a2f678d28e\") " Sep 10 12:26:03.824960 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.824930 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-feast-data" (OuterVolumeSpecName: "feast-data") pod "5a7f8f08-a9fc-4302-8495-f1a2f678d28e" (UID: "5a7f8f08-a9fc-4302-8495-f1a2f678d28e"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 12:26:03.826533 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.826508 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "5a7f8f08-a9fc-4302-8495-f1a2f678d28e" (UID: "5a7f8f08-a9fc-4302-8495-f1a2f678d28e"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:26:03.826696 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.826659 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5a7f8f08-a9fc-4302-8495-f1a2f678d28e" (UID: "5a7f8f08-a9fc-4302-8495-f1a2f678d28e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:26:03.826696 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.826679 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "5a7f8f08-a9fc-4302-8495-f1a2f678d28e" (UID: "5a7f8f08-a9fc-4302-8495-f1a2f678d28e"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:26:03.826696 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.826691 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-kube-api-access-gz6b6" (OuterVolumeSpecName: "kube-api-access-gz6b6") pod "5a7f8f08-a9fc-4302-8495-f1a2f678d28e" (UID: "5a7f8f08-a9fc-4302-8495-f1a2f678d28e"). InnerVolumeSpecName "kube-api-access-gz6b6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:26:03.826944 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.826694 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-online-tls" (OuterVolumeSpecName: "online-tls") pod "5a7f8f08-a9fc-4302-8495-f1a2f678d28e" (UID: "5a7f8f08-a9fc-4302-8495-f1a2f678d28e"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:26:03.829541 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.829525 2580 scope.go:117] "RemoveContainer" containerID="30ad9c82c8aef0aebc90c2e54fd29df6b007711403e9b8c4bdde9225ba1dae5a" Sep 10 12:26:03.838613 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.838594 2580 scope.go:117] "RemoveContainer" containerID="98b5d7036514d0d153f3bd5d340b21aafbcff976a7b3195aaf0c8953ec45dd1d" Sep 10 12:26:03.845799 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.845784 2580 scope.go:117] "RemoveContainer" containerID="f7cc74d160f479f95a826718ef8f8e35e67c879eb416c5d705d46383c47b60b4" Sep 10 12:26:03.846082 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:26:03.846060 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7cc74d160f479f95a826718ef8f8e35e67c879eb416c5d705d46383c47b60b4\": container with ID starting with f7cc74d160f479f95a826718ef8f8e35e67c879eb416c5d705d46383c47b60b4 not found: ID does not exist" containerID="f7cc74d160f479f95a826718ef8f8e35e67c879eb416c5d705d46383c47b60b4" Sep 10 12:26:03.846137 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.846093 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7cc74d160f479f95a826718ef8f8e35e67c879eb416c5d705d46383c47b60b4"} err="failed to get container status \"f7cc74d160f479f95a826718ef8f8e35e67c879eb416c5d705d46383c47b60b4\": rpc error: code = NotFound desc = could not find container \"f7cc74d160f479f95a826718ef8f8e35e67c879eb416c5d705d46383c47b60b4\": container with ID starting with f7cc74d160f479f95a826718ef8f8e35e67c879eb416c5d705d46383c47b60b4 not found: ID does not exist" Sep 10 12:26:03.846137 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.846128 2580 scope.go:117] "RemoveContainer" containerID="1987a05dd027f4e55de68e4bd61572c31a1d6c2a2e8cae26d151793f8c25aa7c" Sep 10 12:26:03.846374 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:26:03.846357 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1987a05dd027f4e55de68e4bd61572c31a1d6c2a2e8cae26d151793f8c25aa7c\": container with ID starting with 1987a05dd027f4e55de68e4bd61572c31a1d6c2a2e8cae26d151793f8c25aa7c not found: ID does not exist" containerID="1987a05dd027f4e55de68e4bd61572c31a1d6c2a2e8cae26d151793f8c25aa7c" Sep 10 12:26:03.846421 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.846380 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1987a05dd027f4e55de68e4bd61572c31a1d6c2a2e8cae26d151793f8c25aa7c"} err="failed to get container status \"1987a05dd027f4e55de68e4bd61572c31a1d6c2a2e8cae26d151793f8c25aa7c\": rpc error: code = NotFound desc = could not find container \"1987a05dd027f4e55de68e4bd61572c31a1d6c2a2e8cae26d151793f8c25aa7c\": container with ID starting with 1987a05dd027f4e55de68e4bd61572c31a1d6c2a2e8cae26d151793f8c25aa7c not found: ID does not exist" Sep 10 12:26:03.846421 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.846396 2580 scope.go:117] "RemoveContainer" containerID="9dda20e8302d2c6c5acfbd110ab8819f0d78f7b279438a626aae92b99069e836" Sep 10 12:26:03.846632 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:26:03.846618 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9dda20e8302d2c6c5acfbd110ab8819f0d78f7b279438a626aae92b99069e836\": container with ID starting with 9dda20e8302d2c6c5acfbd110ab8819f0d78f7b279438a626aae92b99069e836 not found: ID does not exist" containerID="9dda20e8302d2c6c5acfbd110ab8819f0d78f7b279438a626aae92b99069e836" Sep 10 12:26:03.846667 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.846637 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9dda20e8302d2c6c5acfbd110ab8819f0d78f7b279438a626aae92b99069e836"} err="failed to get container status \"9dda20e8302d2c6c5acfbd110ab8819f0d78f7b279438a626aae92b99069e836\": rpc error: code = NotFound desc = could not find container \"9dda20e8302d2c6c5acfbd110ab8819f0d78f7b279438a626aae92b99069e836\": container with ID starting with 9dda20e8302d2c6c5acfbd110ab8819f0d78f7b279438a626aae92b99069e836 not found: ID does not exist" Sep 10 12:26:03.846667 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.846650 2580 scope.go:117] "RemoveContainer" containerID="058063eba96edeec74c71aa0d93c3ba404c5a235a2f4938d8892d27d1ad56039" Sep 10 12:26:03.846971 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:26:03.846851 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"058063eba96edeec74c71aa0d93c3ba404c5a235a2f4938d8892d27d1ad56039\": container with ID starting with 058063eba96edeec74c71aa0d93c3ba404c5a235a2f4938d8892d27d1ad56039 not found: ID does not exist" containerID="058063eba96edeec74c71aa0d93c3ba404c5a235a2f4938d8892d27d1ad56039" Sep 10 12:26:03.847023 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.846975 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"058063eba96edeec74c71aa0d93c3ba404c5a235a2f4938d8892d27d1ad56039"} err="failed to get container status \"058063eba96edeec74c71aa0d93c3ba404c5a235a2f4938d8892d27d1ad56039\": rpc error: code = NotFound desc = could not find container \"058063eba96edeec74c71aa0d93c3ba404c5a235a2f4938d8892d27d1ad56039\": container with ID starting with 058063eba96edeec74c71aa0d93c3ba404c5a235a2f4938d8892d27d1ad56039 not found: ID does not exist" Sep 10 12:26:03.847023 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.846989 2580 scope.go:117] "RemoveContainer" containerID="30ad9c82c8aef0aebc90c2e54fd29df6b007711403e9b8c4bdde9225ba1dae5a" Sep 10 12:26:03.847250 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:26:03.847232 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30ad9c82c8aef0aebc90c2e54fd29df6b007711403e9b8c4bdde9225ba1dae5a\": container with ID starting with 30ad9c82c8aef0aebc90c2e54fd29df6b007711403e9b8c4bdde9225ba1dae5a not found: ID does not exist" containerID="30ad9c82c8aef0aebc90c2e54fd29df6b007711403e9b8c4bdde9225ba1dae5a" Sep 10 12:26:03.847330 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.847251 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30ad9c82c8aef0aebc90c2e54fd29df6b007711403e9b8c4bdde9225ba1dae5a"} err="failed to get container status \"30ad9c82c8aef0aebc90c2e54fd29df6b007711403e9b8c4bdde9225ba1dae5a\": rpc error: code = NotFound desc = could not find container \"30ad9c82c8aef0aebc90c2e54fd29df6b007711403e9b8c4bdde9225ba1dae5a\": container with ID starting with 30ad9c82c8aef0aebc90c2e54fd29df6b007711403e9b8c4bdde9225ba1dae5a not found: ID does not exist" Sep 10 12:26:03.847330 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.847265 2580 scope.go:117] "RemoveContainer" containerID="98b5d7036514d0d153f3bd5d340b21aafbcff976a7b3195aaf0c8953ec45dd1d" Sep 10 12:26:03.847516 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:26:03.847499 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98b5d7036514d0d153f3bd5d340b21aafbcff976a7b3195aaf0c8953ec45dd1d\": container with ID starting with 98b5d7036514d0d153f3bd5d340b21aafbcff976a7b3195aaf0c8953ec45dd1d not found: ID does not exist" containerID="98b5d7036514d0d153f3bd5d340b21aafbcff976a7b3195aaf0c8953ec45dd1d" Sep 10 12:26:03.847596 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.847514 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98b5d7036514d0d153f3bd5d340b21aafbcff976a7b3195aaf0c8953ec45dd1d"} err="failed to get container status \"98b5d7036514d0d153f3bd5d340b21aafbcff976a7b3195aaf0c8953ec45dd1d\": rpc error: code = NotFound desc = could not find container \"98b5d7036514d0d153f3bd5d340b21aafbcff976a7b3195aaf0c8953ec45dd1d\": container with ID starting with 98b5d7036514d0d153f3bd5d340b21aafbcff976a7b3195aaf0c8953ec45dd1d not found: ID does not exist" Sep 10 12:26:03.847596 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.847525 2580 scope.go:117] "RemoveContainer" containerID="f7cc74d160f479f95a826718ef8f8e35e67c879eb416c5d705d46383c47b60b4" Sep 10 12:26:03.847735 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.847718 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7cc74d160f479f95a826718ef8f8e35e67c879eb416c5d705d46383c47b60b4"} err="failed to get container status \"f7cc74d160f479f95a826718ef8f8e35e67c879eb416c5d705d46383c47b60b4\": rpc error: code = NotFound desc = could not find container \"f7cc74d160f479f95a826718ef8f8e35e67c879eb416c5d705d46383c47b60b4\": container with ID starting with f7cc74d160f479f95a826718ef8f8e35e67c879eb416c5d705d46383c47b60b4 not found: ID does not exist" Sep 10 12:26:03.847780 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.847736 2580 scope.go:117] "RemoveContainer" containerID="1987a05dd027f4e55de68e4bd61572c31a1d6c2a2e8cae26d151793f8c25aa7c" Sep 10 12:26:03.847963 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.847945 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1987a05dd027f4e55de68e4bd61572c31a1d6c2a2e8cae26d151793f8c25aa7c"} err="failed to get container status \"1987a05dd027f4e55de68e4bd61572c31a1d6c2a2e8cae26d151793f8c25aa7c\": rpc error: code = NotFound desc = could not find container \"1987a05dd027f4e55de68e4bd61572c31a1d6c2a2e8cae26d151793f8c25aa7c\": container with ID starting with 1987a05dd027f4e55de68e4bd61572c31a1d6c2a2e8cae26d151793f8c25aa7c not found: ID does not exist" Sep 10 12:26:03.848010 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.847963 2580 scope.go:117] "RemoveContainer" containerID="9dda20e8302d2c6c5acfbd110ab8819f0d78f7b279438a626aae92b99069e836" Sep 10 12:26:03.848159 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.848144 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9dda20e8302d2c6c5acfbd110ab8819f0d78f7b279438a626aae92b99069e836"} err="failed to get container status \"9dda20e8302d2c6c5acfbd110ab8819f0d78f7b279438a626aae92b99069e836\": rpc error: code = NotFound desc = could not find container \"9dda20e8302d2c6c5acfbd110ab8819f0d78f7b279438a626aae92b99069e836\": container with ID starting with 9dda20e8302d2c6c5acfbd110ab8819f0d78f7b279438a626aae92b99069e836 not found: ID does not exist" Sep 10 12:26:03.848202 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.848159 2580 scope.go:117] "RemoveContainer" containerID="058063eba96edeec74c71aa0d93c3ba404c5a235a2f4938d8892d27d1ad56039" Sep 10 12:26:03.848362 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.848347 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"058063eba96edeec74c71aa0d93c3ba404c5a235a2f4938d8892d27d1ad56039"} err="failed to get container status \"058063eba96edeec74c71aa0d93c3ba404c5a235a2f4938d8892d27d1ad56039\": rpc error: code = NotFound desc = could not find container \"058063eba96edeec74c71aa0d93c3ba404c5a235a2f4938d8892d27d1ad56039\": container with ID starting with 058063eba96edeec74c71aa0d93c3ba404c5a235a2f4938d8892d27d1ad56039 not found: ID does not exist" Sep 10 12:26:03.848399 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.848363 2580 scope.go:117] "RemoveContainer" containerID="30ad9c82c8aef0aebc90c2e54fd29df6b007711403e9b8c4bdde9225ba1dae5a" Sep 10 12:26:03.848569 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.848550 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30ad9c82c8aef0aebc90c2e54fd29df6b007711403e9b8c4bdde9225ba1dae5a"} err="failed to get container status \"30ad9c82c8aef0aebc90c2e54fd29df6b007711403e9b8c4bdde9225ba1dae5a\": rpc error: code = NotFound desc = could not find container \"30ad9c82c8aef0aebc90c2e54fd29df6b007711403e9b8c4bdde9225ba1dae5a\": container with ID starting with 30ad9c82c8aef0aebc90c2e54fd29df6b007711403e9b8c4bdde9225ba1dae5a not found: ID does not exist" Sep 10 12:26:03.848640 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.848570 2580 scope.go:117] "RemoveContainer" containerID="98b5d7036514d0d153f3bd5d340b21aafbcff976a7b3195aaf0c8953ec45dd1d" Sep 10 12:26:03.848768 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.848751 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98b5d7036514d0d153f3bd5d340b21aafbcff976a7b3195aaf0c8953ec45dd1d"} err="failed to get container status \"98b5d7036514d0d153f3bd5d340b21aafbcff976a7b3195aaf0c8953ec45dd1d\": rpc error: code = NotFound desc = could not find container \"98b5d7036514d0d153f3bd5d340b21aafbcff976a7b3195aaf0c8953ec45dd1d\": container with ID starting with 98b5d7036514d0d153f3bd5d340b21aafbcff976a7b3195aaf0c8953ec45dd1d not found: ID does not exist" Sep 10 12:26:03.925237 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.925207 2580 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-offline-tls\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:26:03.925237 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.925230 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gz6b6\" (UniqueName: \"kubernetes.io/projected/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-kube-api-access-gz6b6\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:26:03.925237 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.925240 2580 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-feast-data\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:26:03.925463 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.925249 2580 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-online-tls\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:26:03.925463 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.925257 2580 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-ui-tls\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:26:03.925463 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:03.925264 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5a7f8f08-a9fc-4302-8495-f1a2f678d28e-registry-tls\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:26:04.116086 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:04.116053 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm"] Sep 10 12:26:04.119977 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:04.119950 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-nvxjm"] Sep 10 12:26:04.557068 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:04.557036 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" path="/var/lib/kubelet/pods/5a7f8f08-a9fc-4302-8495-f1a2f678d28e/volumes" Sep 10 12:26:58.510095 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.510018 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99"] Sep 10 12:26:58.510510 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.510412 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerName="online" Sep 10 12:26:58.510510 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.510426 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerName="online" Sep 10 12:26:58.510510 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.510438 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerName="offline" Sep 10 12:26:58.510510 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.510443 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerName="offline" Sep 10 12:26:58.510510 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.510453 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerName="registry" Sep 10 12:26:58.510510 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.510460 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerName="registry" Sep 10 12:26:58.510510 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.510469 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerName="feast-init" Sep 10 12:26:58.510510 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.510475 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerName="feast-init" Sep 10 12:26:58.510510 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.510482 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerName="feast-apply" Sep 10 12:26:58.510510 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.510486 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerName="feast-apply" Sep 10 12:26:58.510510 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.510494 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerName="ui" Sep 10 12:26:58.510510 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.510499 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerName="ui" Sep 10 12:26:58.510941 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.510555 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerName="ui" Sep 10 12:26:58.510941 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.510563 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerName="offline" Sep 10 12:26:58.510941 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.510568 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerName="online" Sep 10 12:26:58.510941 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.510578 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a7f8f08-a9fc-4302-8495-f1a2f678d28e" containerName="registry" Sep 10 12:26:58.514019 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.514001 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" Sep 10 12:26:58.516900 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.516873 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 10 12:26:58.517036 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.516930 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\"" Sep 10 12:26:58.517036 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.516984 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-vd5rf\"" Sep 10 12:26:58.517162 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.517141 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 10 12:26:58.517217 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.517200 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 10 12:26:58.518188 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.518166 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 10 12:26:58.518304 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.518181 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 10 12:26:58.518304 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.518232 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 10 12:26:58.527940 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.527916 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99"] Sep 10 12:26:58.604434 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.604407 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/398fbfd2-4051-4c06-a844-06f4def11fd2-ui-tls\") pod \"feast-simple-feast-remote-setup-665c797886-6kt99\" (UID: \"398fbfd2-4051-4c06-a844-06f4def11fd2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" Sep 10 12:26:58.604592 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.604463 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/398fbfd2-4051-4c06-a844-06f4def11fd2-offline-tls\") pod \"feast-simple-feast-remote-setup-665c797886-6kt99\" (UID: \"398fbfd2-4051-4c06-a844-06f4def11fd2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" Sep 10 12:26:58.604592 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.604513 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgkr6\" (UniqueName: \"kubernetes.io/projected/398fbfd2-4051-4c06-a844-06f4def11fd2-kube-api-access-bgkr6\") pod \"feast-simple-feast-remote-setup-665c797886-6kt99\" (UID: \"398fbfd2-4051-4c06-a844-06f4def11fd2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" Sep 10 12:26:58.604592 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.604538 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/398fbfd2-4051-4c06-a844-06f4def11fd2-online-tls\") pod \"feast-simple-feast-remote-setup-665c797886-6kt99\" (UID: \"398fbfd2-4051-4c06-a844-06f4def11fd2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" Sep 10 12:26:58.604720 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.604655 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/398fbfd2-4051-4c06-a844-06f4def11fd2-feast-data\") pod \"feast-simple-feast-remote-setup-665c797886-6kt99\" (UID: \"398fbfd2-4051-4c06-a844-06f4def11fd2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" Sep 10 12:26:58.604755 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.604716 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/398fbfd2-4051-4c06-a844-06f4def11fd2-registry-tls\") pod \"feast-simple-feast-remote-setup-665c797886-6kt99\" (UID: \"398fbfd2-4051-4c06-a844-06f4def11fd2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" Sep 10 12:26:58.706017 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.705972 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/398fbfd2-4051-4c06-a844-06f4def11fd2-online-tls\") pod \"feast-simple-feast-remote-setup-665c797886-6kt99\" (UID: \"398fbfd2-4051-4c06-a844-06f4def11fd2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" Sep 10 12:26:58.706017 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.706020 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/398fbfd2-4051-4c06-a844-06f4def11fd2-feast-data\") pod \"feast-simple-feast-remote-setup-665c797886-6kt99\" (UID: \"398fbfd2-4051-4c06-a844-06f4def11fd2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" Sep 10 12:26:58.706274 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.706058 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/398fbfd2-4051-4c06-a844-06f4def11fd2-registry-tls\") pod \"feast-simple-feast-remote-setup-665c797886-6kt99\" (UID: \"398fbfd2-4051-4c06-a844-06f4def11fd2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" Sep 10 12:26:58.706274 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.706089 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/398fbfd2-4051-4c06-a844-06f4def11fd2-ui-tls\") pod \"feast-simple-feast-remote-setup-665c797886-6kt99\" (UID: \"398fbfd2-4051-4c06-a844-06f4def11fd2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" Sep 10 12:26:58.706274 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:26:58.706111 2580 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-online-tls: secret "feast-simple-feast-remote-setup-online-tls" not found Sep 10 12:26:58.706274 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.706139 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/398fbfd2-4051-4c06-a844-06f4def11fd2-offline-tls\") pod \"feast-simple-feast-remote-setup-665c797886-6kt99\" (UID: \"398fbfd2-4051-4c06-a844-06f4def11fd2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" Sep 10 12:26:58.706274 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:26:58.706173 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/398fbfd2-4051-4c06-a844-06f4def11fd2-online-tls podName:398fbfd2-4051-4c06-a844-06f4def11fd2 nodeName:}" failed. No retries permitted until 2026-09-10 12:26:59.206152392 +0000 UTC m=+539.213468757 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/398fbfd2-4051-4c06-a844-06f4def11fd2-online-tls") pod "feast-simple-feast-remote-setup-665c797886-6kt99" (UID: "398fbfd2-4051-4c06-a844-06f4def11fd2") : secret "feast-simple-feast-remote-setup-online-tls" not found Sep 10 12:26:58.706274 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.706208 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgkr6\" (UniqueName: \"kubernetes.io/projected/398fbfd2-4051-4c06-a844-06f4def11fd2-kube-api-access-bgkr6\") pod \"feast-simple-feast-remote-setup-665c797886-6kt99\" (UID: \"398fbfd2-4051-4c06-a844-06f4def11fd2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" Sep 10 12:26:58.706615 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.706500 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/398fbfd2-4051-4c06-a844-06f4def11fd2-feast-data\") pod \"feast-simple-feast-remote-setup-665c797886-6kt99\" (UID: \"398fbfd2-4051-4c06-a844-06f4def11fd2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" Sep 10 12:26:58.706834 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.706815 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/398fbfd2-4051-4c06-a844-06f4def11fd2-registry-tls\") pod \"feast-simple-feast-remote-setup-665c797886-6kt99\" (UID: \"398fbfd2-4051-4c06-a844-06f4def11fd2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" Sep 10 12:26:58.708502 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.708480 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/398fbfd2-4051-4c06-a844-06f4def11fd2-ui-tls\") pod \"feast-simple-feast-remote-setup-665c797886-6kt99\" (UID: \"398fbfd2-4051-4c06-a844-06f4def11fd2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" Sep 10 12:26:58.708644 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.708627 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/398fbfd2-4051-4c06-a844-06f4def11fd2-offline-tls\") pod \"feast-simple-feast-remote-setup-665c797886-6kt99\" (UID: \"398fbfd2-4051-4c06-a844-06f4def11fd2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" Sep 10 12:26:58.715847 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:58.715825 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgkr6\" (UniqueName: \"kubernetes.io/projected/398fbfd2-4051-4c06-a844-06f4def11fd2-kube-api-access-bgkr6\") pod \"feast-simple-feast-remote-setup-665c797886-6kt99\" (UID: \"398fbfd2-4051-4c06-a844-06f4def11fd2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" Sep 10 12:26:59.210547 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:59.210515 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/398fbfd2-4051-4c06-a844-06f4def11fd2-online-tls\") pod \"feast-simple-feast-remote-setup-665c797886-6kt99\" (UID: \"398fbfd2-4051-4c06-a844-06f4def11fd2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" Sep 10 12:26:59.212975 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:59.212951 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/398fbfd2-4051-4c06-a844-06f4def11fd2-online-tls\") pod \"feast-simple-feast-remote-setup-665c797886-6kt99\" (UID: \"398fbfd2-4051-4c06-a844-06f4def11fd2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" Sep 10 12:26:59.423744 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:59.423692 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" Sep 10 12:26:59.701329 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:26:59.701296 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99"] Sep 10 12:26:59.704309 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:26:59.704283 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod398fbfd2_4051_4c06_a844_06f4def11fd2.slice/crio-ee3d9fd0eb549d419fa9b62d7600f6f60fa264f30c582fcf5ad5ce63f5b065ed WatchSource:0}: Error finding container ee3d9fd0eb549d419fa9b62d7600f6f60fa264f30c582fcf5ad5ce63f5b065ed: Status 404 returned error can't find the container with id ee3d9fd0eb549d419fa9b62d7600f6f60fa264f30c582fcf5ad5ce63f5b065ed Sep 10 12:27:00.029845 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:27:00.029756 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" event={"ID":"398fbfd2-4051-4c06-a844-06f4def11fd2","Type":"ContainerStarted","Data":"d3a03fcba239265e913f999b9820e0c38a6d3d65fca2f3e2532b4d167fd0b9cd"} Sep 10 12:27:00.030090 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:27:00.030073 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" event={"ID":"398fbfd2-4051-4c06-a844-06f4def11fd2","Type":"ContainerStarted","Data":"ee3d9fd0eb549d419fa9b62d7600f6f60fa264f30c582fcf5ad5ce63f5b065ed"} Sep 10 12:27:05.053132 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:27:05.053100 2580 generic.go:356] "Generic (PLEG): container finished" podID="398fbfd2-4051-4c06-a844-06f4def11fd2" containerID="d3a03fcba239265e913f999b9820e0c38a6d3d65fca2f3e2532b4d167fd0b9cd" exitCode=0 Sep 10 12:27:05.053551 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:27:05.053170 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" event={"ID":"398fbfd2-4051-4c06-a844-06f4def11fd2","Type":"ContainerDied","Data":"d3a03fcba239265e913f999b9820e0c38a6d3d65fca2f3e2532b4d167fd0b9cd"} Sep 10 12:27:06.059475 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:27:06.059441 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" event={"ID":"398fbfd2-4051-4c06-a844-06f4def11fd2","Type":"ContainerStarted","Data":"2afbb48fb203331572e9e4abf8812767aeed58908c8cd233bd7a1144cedd5a4a"} Sep 10 12:27:28.151896 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:27:28.151848 2580 generic.go:356] "Generic (PLEG): container finished" podID="398fbfd2-4051-4c06-a844-06f4def11fd2" containerID="2afbb48fb203331572e9e4abf8812767aeed58908c8cd233bd7a1144cedd5a4a" exitCode=0 Sep 10 12:27:28.152263 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:27:28.151922 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" event={"ID":"398fbfd2-4051-4c06-a844-06f4def11fd2","Type":"ContainerDied","Data":"2afbb48fb203331572e9e4abf8812767aeed58908c8cd233bd7a1144cedd5a4a"} Sep 10 12:27:28.155686 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:27:28.155662 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\"" Sep 10 12:27:29.161606 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:27:29.161565 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" event={"ID":"398fbfd2-4051-4c06-a844-06f4def11fd2","Type":"ContainerStarted","Data":"4e2b9e75aa29a2210d8163eeae64c08eaf909049d0f4626b0dde5f9c9bc85fb9"} Sep 10 12:27:29.162013 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:27:29.161625 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" event={"ID":"398fbfd2-4051-4c06-a844-06f4def11fd2","Type":"ContainerStarted","Data":"ea9e15b5d9962d0f2130356197cd29267d261b9fb717a96df2e40b009781e66c"} Sep 10 12:27:29.162013 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:27:29.161646 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" event={"ID":"398fbfd2-4051-4c06-a844-06f4def11fd2","Type":"ContainerStarted","Data":"62fd9fd1fcc3134d6c204ba46a260afaaa70115a45cbdeec532003325fb86f47"} Sep 10 12:27:29.191718 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:27:29.191656 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" podStartSLOduration=31.191633196 podStartE2EDuration="31.191633196s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:26:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:27:29.191466369 +0000 UTC m=+569.198782776" watchObservedRunningTime="2026-09-10 12:27:29.191633196 +0000 UTC m=+569.198949582" Sep 10 12:27:29.424799 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:27:29.424714 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" Sep 10 12:27:29.424799 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:27:29.424769 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" Sep 10 12:27:29.424799 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:27:29.424788 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" Sep 10 12:27:29.424799 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:27:29.424804 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" Sep 10 12:27:29.425141 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:27:29.424820 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" Sep 10 12:27:29.425141 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:27:29.424834 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" Sep 10 12:27:29.426759 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:27:29.426722 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" podUID="398fbfd2-4051-4c06-a844-06f4def11fd2" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.22:8816: connect: connection refused" Sep 10 12:27:29.426914 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:27:29.426745 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" podUID="398fbfd2-4051-4c06-a844-06f4def11fd2" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.22:8443: connect: connection refused" Sep 10 12:27:29.426914 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:27:29.426879 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" podUID="398fbfd2-4051-4c06-a844-06f4def11fd2" containerName="online" probeResult="failure" output="Get \"https://10.134.0.22:6567/health\": dial tcp 10.134.0.22:6567: connect: connection refused" Sep 10 12:27:32.424331 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:27:32.424280 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" podUID="398fbfd2-4051-4c06-a844-06f4def11fd2" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.22:8816: connect: connection refused" Sep 10 12:27:32.424807 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:27:32.424362 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" podUID="398fbfd2-4051-4c06-a844-06f4def11fd2" containerName="online" probeResult="failure" output="Get \"https://10.134.0.22:6567/health\": dial tcp 10.134.0.22:6567: connect: connection refused" Sep 10 12:27:32.424807 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:27:32.424294 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" podUID="398fbfd2-4051-4c06-a844-06f4def11fd2" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.22:8443: connect: connection refused" Sep 10 12:27:35.424257 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:27:35.424210 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" podUID="398fbfd2-4051-4c06-a844-06f4def11fd2" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.22:8443: connect: connection refused" Sep 10 12:27:35.424715 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:27:35.424257 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" podUID="398fbfd2-4051-4c06-a844-06f4def11fd2" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.22:8816: connect: connection refused" Sep 10 12:27:35.424715 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:27:35.424289 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" podUID="398fbfd2-4051-4c06-a844-06f4def11fd2" containerName="online" probeResult="failure" output="Get \"https://10.134.0.22:6567/health\": dial tcp 10.134.0.22:6567: connect: connection refused" Sep 10 12:27:38.425302 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:27:38.425272 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" Sep 10 12:27:38.425794 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:27:38.425772 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" Sep 10 12:27:38.426006 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:27:38.425985 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" Sep 10 12:27:38.446998 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:27:38.446972 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" Sep 10 12:27:39.217191 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:27:39.217152 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" Sep 10 12:27:39.235035 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:27:39.235009 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" Sep 10 12:28:00.448477 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:28:00.448448 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tfcmz_e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58/ovn-acl-logging/0.log" Sep 10 12:28:00.449248 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:28:00.449233 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tfcmz_e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58/ovn-acl-logging/0.log" Sep 10 12:28:40.747161 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:28:40.746769 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99"] Sep 10 12:28:40.747161 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:28:40.747070 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" podUID="398fbfd2-4051-4c06-a844-06f4def11fd2" containerName="online" containerID="cri-o://62fd9fd1fcc3134d6c204ba46a260afaaa70115a45cbdeec532003325fb86f47" gracePeriod=30 Sep 10 12:28:40.749713 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:28:40.747205 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" podUID="398fbfd2-4051-4c06-a844-06f4def11fd2" containerName="offline" containerID="cri-o://ea9e15b5d9962d0f2130356197cd29267d261b9fb717a96df2e40b009781e66c" gracePeriod=30 Sep 10 12:28:40.749713 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:28:40.747309 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" podUID="398fbfd2-4051-4c06-a844-06f4def11fd2" containerName="ui" containerID="cri-o://4e2b9e75aa29a2210d8163eeae64c08eaf909049d0f4626b0dde5f9c9bc85fb9" gracePeriod=30 Sep 10 12:28:40.872898 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:28:40.872849 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9"] Sep 10 12:28:40.873344 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:28:40.873149 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" podUID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerName="registry" containerID="cri-o://dc781d101bc450900821fc6902d3e255222ece7253b0ab8eb37d413d4428729b" gracePeriod=30 Sep 10 12:28:40.873344 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:28:40.873223 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" podUID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerName="ui" containerID="cri-o://dc87ea858365977bfcf8ea10957a368e05e2fe969dfbadf57aa8b3ef8c395d76" gracePeriod=30 Sep 10 12:28:40.873344 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:28:40.873261 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" podUID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerName="online" containerID="cri-o://be86f7ec3ce31061678962a95c92729fede788683b97b8e0fdfece51d6be0676" gracePeriod=30 Sep 10 12:28:40.873344 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:28:40.873286 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" podUID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerName="offline" containerID="cri-o://f2a1c93943977c8ab16d32cc23ebeae29b0ba06520b1bf97f4ad16b10793c5f3" gracePeriod=30 Sep 10 12:28:41.472779 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:28:41.472747 2580 generic.go:356] "Generic (PLEG): container finished" podID="398fbfd2-4051-4c06-a844-06f4def11fd2" containerID="4e2b9e75aa29a2210d8163eeae64c08eaf909049d0f4626b0dde5f9c9bc85fb9" exitCode=0 Sep 10 12:28:41.472992 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:28:41.472822 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" event={"ID":"398fbfd2-4051-4c06-a844-06f4def11fd2","Type":"ContainerDied","Data":"4e2b9e75aa29a2210d8163eeae64c08eaf909049d0f4626b0dde5f9c9bc85fb9"} Sep 10 12:28:42.480301 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:28:42.480269 2580 generic.go:356] "Generic (PLEG): container finished" podID="398fbfd2-4051-4c06-a844-06f4def11fd2" containerID="62fd9fd1fcc3134d6c204ba46a260afaaa70115a45cbdeec532003325fb86f47" exitCode=0 Sep 10 12:28:42.480719 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:28:42.480346 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" event={"ID":"398fbfd2-4051-4c06-a844-06f4def11fd2","Type":"ContainerDied","Data":"62fd9fd1fcc3134d6c204ba46a260afaaa70115a45cbdeec532003325fb86f47"} Sep 10 12:28:42.483052 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:28:42.483030 2580 generic.go:356] "Generic (PLEG): container finished" podID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerID="dc87ea858365977bfcf8ea10957a368e05e2fe969dfbadf57aa8b3ef8c395d76" exitCode=0 Sep 10 12:28:42.483052 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:28:42.483048 2580 generic.go:356] "Generic (PLEG): container finished" podID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerID="be86f7ec3ce31061678962a95c92729fede788683b97b8e0fdfece51d6be0676" exitCode=0 Sep 10 12:28:42.483197 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:28:42.483071 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" event={"ID":"0570e6d2-af94-447b-a8e2-73b63855d2ee","Type":"ContainerDied","Data":"dc87ea858365977bfcf8ea10957a368e05e2fe969dfbadf57aa8b3ef8c395d76"} Sep 10 12:28:42.483197 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:28:42.483093 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" event={"ID":"0570e6d2-af94-447b-a8e2-73b63855d2ee","Type":"ContainerDied","Data":"be86f7ec3ce31061678962a95c92729fede788683b97b8e0fdfece51d6be0676"} Sep 10 12:28:45.369819 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:28:45.369780 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" podUID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.21:8443: connect: connection refused" Sep 10 12:28:45.753765 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:28:45.753731 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" podUID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerName="online" probeResult="failure" output="Get \"https://10.134.0.21:6567/health\": dial tcp 10.134.0.21:6567: connect: connection refused" Sep 10 12:28:48.426724 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:28:48.426683 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" podUID="398fbfd2-4051-4c06-a844-06f4def11fd2" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.22:8443: connect: connection refused" Sep 10 12:28:49.215738 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:28:49.215691 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" podUID="398fbfd2-4051-4c06-a844-06f4def11fd2" containerName="online" probeResult="failure" output="Get \"https://10.134.0.22:6567/health\": dial tcp 10.134.0.22:6567: connect: connection refused" Sep 10 12:28:55.369705 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:28:55.369650 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" podUID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.21:8443: connect: connection refused" Sep 10 12:28:55.753799 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:28:55.753756 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" podUID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerName="online" probeResult="failure" output="Get \"https://10.134.0.21:6567/health\": dial tcp 10.134.0.21:6567: connect: connection refused" Sep 10 12:28:58.426143 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:28:58.426101 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" podUID="398fbfd2-4051-4c06-a844-06f4def11fd2" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.22:8443: connect: connection refused" Sep 10 12:28:59.215250 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:28:59.215204 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" podUID="398fbfd2-4051-4c06-a844-06f4def11fd2" containerName="online" probeResult="failure" output="Get \"https://10.134.0.22:6567/health\": dial tcp 10.134.0.22:6567: connect: connection refused" Sep 10 12:29:05.369243 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:05.369193 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" podUID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.21:8443: connect: connection refused" Sep 10 12:29:05.369701 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:05.369385 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:29:05.754073 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:05.754029 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" podUID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerName="online" probeResult="failure" output="Get \"https://10.134.0.21:6567/health\": dial tcp 10.134.0.21:6567: connect: connection refused" Sep 10 12:29:05.754239 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:05.754143 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:29:08.425805 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:08.425765 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" podUID="398fbfd2-4051-4c06-a844-06f4def11fd2" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.22:8443: connect: connection refused" Sep 10 12:29:08.426209 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:08.425906 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" Sep 10 12:29:09.215783 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:09.215745 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" podUID="398fbfd2-4051-4c06-a844-06f4def11fd2" containerName="online" probeResult="failure" output="Get \"https://10.134.0.22:6567/health\": dial tcp 10.134.0.22:6567: connect: connection refused" Sep 10 12:29:09.215991 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:09.215911 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" Sep 10 12:29:11.389335 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.389315 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" Sep 10 12:29:11.477375 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.477340 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bgkr6\" (UniqueName: \"kubernetes.io/projected/398fbfd2-4051-4c06-a844-06f4def11fd2-kube-api-access-bgkr6\") pod \"398fbfd2-4051-4c06-a844-06f4def11fd2\" (UID: \"398fbfd2-4051-4c06-a844-06f4def11fd2\") " Sep 10 12:29:11.477538 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.477395 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/398fbfd2-4051-4c06-a844-06f4def11fd2-offline-tls\") pod \"398fbfd2-4051-4c06-a844-06f4def11fd2\" (UID: \"398fbfd2-4051-4c06-a844-06f4def11fd2\") " Sep 10 12:29:11.477538 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.477476 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/398fbfd2-4051-4c06-a844-06f4def11fd2-online-tls\") pod \"398fbfd2-4051-4c06-a844-06f4def11fd2\" (UID: \"398fbfd2-4051-4c06-a844-06f4def11fd2\") " Sep 10 12:29:11.477538 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.477499 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/398fbfd2-4051-4c06-a844-06f4def11fd2-registry-tls\") pod \"398fbfd2-4051-4c06-a844-06f4def11fd2\" (UID: \"398fbfd2-4051-4c06-a844-06f4def11fd2\") " Sep 10 12:29:11.477707 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.477543 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/398fbfd2-4051-4c06-a844-06f4def11fd2-ui-tls\") pod \"398fbfd2-4051-4c06-a844-06f4def11fd2\" (UID: \"398fbfd2-4051-4c06-a844-06f4def11fd2\") " Sep 10 12:29:11.477707 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.477580 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/398fbfd2-4051-4c06-a844-06f4def11fd2-feast-data\") pod \"398fbfd2-4051-4c06-a844-06f4def11fd2\" (UID: \"398fbfd2-4051-4c06-a844-06f4def11fd2\") " Sep 10 12:29:11.477935 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.477897 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/398fbfd2-4051-4c06-a844-06f4def11fd2-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "398fbfd2-4051-4c06-a844-06f4def11fd2" (UID: "398fbfd2-4051-4c06-a844-06f4def11fd2"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:29:11.478283 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.478251 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/398fbfd2-4051-4c06-a844-06f4def11fd2-feast-data" (OuterVolumeSpecName: "feast-data") pod "398fbfd2-4051-4c06-a844-06f4def11fd2" (UID: "398fbfd2-4051-4c06-a844-06f4def11fd2"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 12:29:11.481377 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.481349 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/398fbfd2-4051-4c06-a844-06f4def11fd2-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "398fbfd2-4051-4c06-a844-06f4def11fd2" (UID: "398fbfd2-4051-4c06-a844-06f4def11fd2"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:29:11.481476 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.481390 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/398fbfd2-4051-4c06-a844-06f4def11fd2-online-tls" (OuterVolumeSpecName: "online-tls") pod "398fbfd2-4051-4c06-a844-06f4def11fd2" (UID: "398fbfd2-4051-4c06-a844-06f4def11fd2"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:29:11.481476 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.481425 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/398fbfd2-4051-4c06-a844-06f4def11fd2-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "398fbfd2-4051-4c06-a844-06f4def11fd2" (UID: "398fbfd2-4051-4c06-a844-06f4def11fd2"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:29:11.481476 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.481440 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/398fbfd2-4051-4c06-a844-06f4def11fd2-kube-api-access-bgkr6" (OuterVolumeSpecName: "kube-api-access-bgkr6") pod "398fbfd2-4051-4c06-a844-06f4def11fd2" (UID: "398fbfd2-4051-4c06-a844-06f4def11fd2"). InnerVolumeSpecName "kube-api-access-bgkr6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:29:11.502425 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.502405 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:29:11.578953 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.578875 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0570e6d2-af94-447b-a8e2-73b63855d2ee-ui-tls\") pod \"0570e6d2-af94-447b-a8e2-73b63855d2ee\" (UID: \"0570e6d2-af94-447b-a8e2-73b63855d2ee\") " Sep 10 12:29:11.578953 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.578910 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0570e6d2-af94-447b-a8e2-73b63855d2ee-online-tls\") pod \"0570e6d2-af94-447b-a8e2-73b63855d2ee\" (UID: \"0570e6d2-af94-447b-a8e2-73b63855d2ee\") " Sep 10 12:29:11.578953 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.578944 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0570e6d2-af94-447b-a8e2-73b63855d2ee-feast-data\") pod \"0570e6d2-af94-447b-a8e2-73b63855d2ee\" (UID: \"0570e6d2-af94-447b-a8e2-73b63855d2ee\") " Sep 10 12:29:11.579213 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.579057 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mqv7h\" (UniqueName: \"kubernetes.io/projected/0570e6d2-af94-447b-a8e2-73b63855d2ee-kube-api-access-mqv7h\") pod \"0570e6d2-af94-447b-a8e2-73b63855d2ee\" (UID: \"0570e6d2-af94-447b-a8e2-73b63855d2ee\") " Sep 10 12:29:11.579213 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.579101 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0570e6d2-af94-447b-a8e2-73b63855d2ee-offline-tls\") pod \"0570e6d2-af94-447b-a8e2-73b63855d2ee\" (UID: \"0570e6d2-af94-447b-a8e2-73b63855d2ee\") " Sep 10 12:29:11.579213 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.579130 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0570e6d2-af94-447b-a8e2-73b63855d2ee-registry-tls\") pod \"0570e6d2-af94-447b-a8e2-73b63855d2ee\" (UID: \"0570e6d2-af94-447b-a8e2-73b63855d2ee\") " Sep 10 12:29:11.579500 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.579445 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0570e6d2-af94-447b-a8e2-73b63855d2ee-feast-data" (OuterVolumeSpecName: "feast-data") pod "0570e6d2-af94-447b-a8e2-73b63855d2ee" (UID: "0570e6d2-af94-447b-a8e2-73b63855d2ee"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 12:29:11.579500 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.579454 2580 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/398fbfd2-4051-4c06-a844-06f4def11fd2-ui-tls\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:29:11.579500 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.579493 2580 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/398fbfd2-4051-4c06-a844-06f4def11fd2-feast-data\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:29:11.579705 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.579510 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgkr6\" (UniqueName: \"kubernetes.io/projected/398fbfd2-4051-4c06-a844-06f4def11fd2-kube-api-access-bgkr6\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:29:11.579705 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.579523 2580 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/398fbfd2-4051-4c06-a844-06f4def11fd2-offline-tls\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:29:11.579705 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.579558 2580 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/398fbfd2-4051-4c06-a844-06f4def11fd2-online-tls\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:29:11.579705 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.579571 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/398fbfd2-4051-4c06-a844-06f4def11fd2-registry-tls\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:29:11.581176 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.581148 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0570e6d2-af94-447b-a8e2-73b63855d2ee-kube-api-access-mqv7h" (OuterVolumeSpecName: "kube-api-access-mqv7h") pod "0570e6d2-af94-447b-a8e2-73b63855d2ee" (UID: "0570e6d2-af94-447b-a8e2-73b63855d2ee"). InnerVolumeSpecName "kube-api-access-mqv7h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:29:11.581443 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.581425 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0570e6d2-af94-447b-a8e2-73b63855d2ee-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "0570e6d2-af94-447b-a8e2-73b63855d2ee" (UID: "0570e6d2-af94-447b-a8e2-73b63855d2ee"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:29:11.581527 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.581442 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0570e6d2-af94-447b-a8e2-73b63855d2ee-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "0570e6d2-af94-447b-a8e2-73b63855d2ee" (UID: "0570e6d2-af94-447b-a8e2-73b63855d2ee"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:29:11.581527 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.581511 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0570e6d2-af94-447b-a8e2-73b63855d2ee-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "0570e6d2-af94-447b-a8e2-73b63855d2ee" (UID: "0570e6d2-af94-447b-a8e2-73b63855d2ee"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:29:11.581590 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.581532 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0570e6d2-af94-447b-a8e2-73b63855d2ee-online-tls" (OuterVolumeSpecName: "online-tls") pod "0570e6d2-af94-447b-a8e2-73b63855d2ee" (UID: "0570e6d2-af94-447b-a8e2-73b63855d2ee"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:29:11.611146 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.611120 2580 generic.go:356] "Generic (PLEG): container finished" podID="398fbfd2-4051-4c06-a844-06f4def11fd2" containerID="ea9e15b5d9962d0f2130356197cd29267d261b9fb717a96df2e40b009781e66c" exitCode=137 Sep 10 12:29:11.611251 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.611191 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" Sep 10 12:29:11.611251 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.611196 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" event={"ID":"398fbfd2-4051-4c06-a844-06f4def11fd2","Type":"ContainerDied","Data":"ea9e15b5d9962d0f2130356197cd29267d261b9fb717a96df2e40b009781e66c"} Sep 10 12:29:11.611376 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.611249 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99" event={"ID":"398fbfd2-4051-4c06-a844-06f4def11fd2","Type":"ContainerDied","Data":"ee3d9fd0eb549d419fa9b62d7600f6f60fa264f30c582fcf5ad5ce63f5b065ed"} Sep 10 12:29:11.611376 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.611278 2580 scope.go:117] "RemoveContainer" containerID="4e2b9e75aa29a2210d8163eeae64c08eaf909049d0f4626b0dde5f9c9bc85fb9" Sep 10 12:29:11.614529 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.614504 2580 generic.go:356] "Generic (PLEG): container finished" podID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerID="f2a1c93943977c8ab16d32cc23ebeae29b0ba06520b1bf97f4ad16b10793c5f3" exitCode=137 Sep 10 12:29:11.614621 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.614531 2580 generic.go:356] "Generic (PLEG): container finished" podID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerID="dc781d101bc450900821fc6902d3e255222ece7253b0ab8eb37d413d4428729b" exitCode=137 Sep 10 12:29:11.614621 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.614530 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" event={"ID":"0570e6d2-af94-447b-a8e2-73b63855d2ee","Type":"ContainerDied","Data":"f2a1c93943977c8ab16d32cc23ebeae29b0ba06520b1bf97f4ad16b10793c5f3"} Sep 10 12:29:11.614621 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.614570 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" event={"ID":"0570e6d2-af94-447b-a8e2-73b63855d2ee","Type":"ContainerDied","Data":"dc781d101bc450900821fc6902d3e255222ece7253b0ab8eb37d413d4428729b"} Sep 10 12:29:11.614621 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.614591 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" event={"ID":"0570e6d2-af94-447b-a8e2-73b63855d2ee","Type":"ContainerDied","Data":"4b8469ea65f590e1d9b8c65bd1d81fe55fee307f6937c02d66118a45e1f0c253"} Sep 10 12:29:11.614621 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.614592 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9" Sep 10 12:29:11.620093 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.620077 2580 scope.go:117] "RemoveContainer" containerID="ea9e15b5d9962d0f2130356197cd29267d261b9fb717a96df2e40b009781e66c" Sep 10 12:29:11.628124 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.628103 2580 scope.go:117] "RemoveContainer" containerID="62fd9fd1fcc3134d6c204ba46a260afaaa70115a45cbdeec532003325fb86f47" Sep 10 12:29:11.633503 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.633479 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99"] Sep 10 12:29:11.635588 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.635575 2580 scope.go:117] "RemoveContainer" containerID="2afbb48fb203331572e9e4abf8812767aeed58908c8cd233bd7a1144cedd5a4a" Sep 10 12:29:11.639673 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.639651 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-665c797886-6kt99"] Sep 10 12:29:11.642935 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.642913 2580 scope.go:117] "RemoveContainer" containerID="d3a03fcba239265e913f999b9820e0c38a6d3d65fca2f3e2532b4d167fd0b9cd" Sep 10 12:29:11.649821 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.649806 2580 scope.go:117] "RemoveContainer" containerID="4e2b9e75aa29a2210d8163eeae64c08eaf909049d0f4626b0dde5f9c9bc85fb9" Sep 10 12:29:11.650102 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:29:11.650084 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e2b9e75aa29a2210d8163eeae64c08eaf909049d0f4626b0dde5f9c9bc85fb9\": container with ID starting with 4e2b9e75aa29a2210d8163eeae64c08eaf909049d0f4626b0dde5f9c9bc85fb9 not found: ID does not exist" containerID="4e2b9e75aa29a2210d8163eeae64c08eaf909049d0f4626b0dde5f9c9bc85fb9" Sep 10 12:29:11.650183 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.650111 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e2b9e75aa29a2210d8163eeae64c08eaf909049d0f4626b0dde5f9c9bc85fb9"} err="failed to get container status \"4e2b9e75aa29a2210d8163eeae64c08eaf909049d0f4626b0dde5f9c9bc85fb9\": rpc error: code = NotFound desc = could not find container \"4e2b9e75aa29a2210d8163eeae64c08eaf909049d0f4626b0dde5f9c9bc85fb9\": container with ID starting with 4e2b9e75aa29a2210d8163eeae64c08eaf909049d0f4626b0dde5f9c9bc85fb9 not found: ID does not exist" Sep 10 12:29:11.650183 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.650128 2580 scope.go:117] "RemoveContainer" containerID="ea9e15b5d9962d0f2130356197cd29267d261b9fb717a96df2e40b009781e66c" Sep 10 12:29:11.650361 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:29:11.650344 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea9e15b5d9962d0f2130356197cd29267d261b9fb717a96df2e40b009781e66c\": container with ID starting with ea9e15b5d9962d0f2130356197cd29267d261b9fb717a96df2e40b009781e66c not found: ID does not exist" containerID="ea9e15b5d9962d0f2130356197cd29267d261b9fb717a96df2e40b009781e66c" Sep 10 12:29:11.650416 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.650369 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea9e15b5d9962d0f2130356197cd29267d261b9fb717a96df2e40b009781e66c"} err="failed to get container status \"ea9e15b5d9962d0f2130356197cd29267d261b9fb717a96df2e40b009781e66c\": rpc error: code = NotFound desc = could not find container \"ea9e15b5d9962d0f2130356197cd29267d261b9fb717a96df2e40b009781e66c\": container with ID starting with ea9e15b5d9962d0f2130356197cd29267d261b9fb717a96df2e40b009781e66c not found: ID does not exist" Sep 10 12:29:11.650416 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.650391 2580 scope.go:117] "RemoveContainer" containerID="62fd9fd1fcc3134d6c204ba46a260afaaa70115a45cbdeec532003325fb86f47" Sep 10 12:29:11.650626 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:29:11.650609 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62fd9fd1fcc3134d6c204ba46a260afaaa70115a45cbdeec532003325fb86f47\": container with ID starting with 62fd9fd1fcc3134d6c204ba46a260afaaa70115a45cbdeec532003325fb86f47 not found: ID does not exist" containerID="62fd9fd1fcc3134d6c204ba46a260afaaa70115a45cbdeec532003325fb86f47" Sep 10 12:29:11.650671 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.650633 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62fd9fd1fcc3134d6c204ba46a260afaaa70115a45cbdeec532003325fb86f47"} err="failed to get container status \"62fd9fd1fcc3134d6c204ba46a260afaaa70115a45cbdeec532003325fb86f47\": rpc error: code = NotFound desc = could not find container \"62fd9fd1fcc3134d6c204ba46a260afaaa70115a45cbdeec532003325fb86f47\": container with ID starting with 62fd9fd1fcc3134d6c204ba46a260afaaa70115a45cbdeec532003325fb86f47 not found: ID does not exist" Sep 10 12:29:11.650671 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.650653 2580 scope.go:117] "RemoveContainer" containerID="2afbb48fb203331572e9e4abf8812767aeed58908c8cd233bd7a1144cedd5a4a" Sep 10 12:29:11.650889 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:29:11.650869 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2afbb48fb203331572e9e4abf8812767aeed58908c8cd233bd7a1144cedd5a4a\": container with ID starting with 2afbb48fb203331572e9e4abf8812767aeed58908c8cd233bd7a1144cedd5a4a not found: ID does not exist" containerID="2afbb48fb203331572e9e4abf8812767aeed58908c8cd233bd7a1144cedd5a4a" Sep 10 12:29:11.650933 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.650892 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2afbb48fb203331572e9e4abf8812767aeed58908c8cd233bd7a1144cedd5a4a"} err="failed to get container status \"2afbb48fb203331572e9e4abf8812767aeed58908c8cd233bd7a1144cedd5a4a\": rpc error: code = NotFound desc = could not find container \"2afbb48fb203331572e9e4abf8812767aeed58908c8cd233bd7a1144cedd5a4a\": container with ID starting with 2afbb48fb203331572e9e4abf8812767aeed58908c8cd233bd7a1144cedd5a4a not found: ID does not exist" Sep 10 12:29:11.650933 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.650904 2580 scope.go:117] "RemoveContainer" containerID="d3a03fcba239265e913f999b9820e0c38a6d3d65fca2f3e2532b4d167fd0b9cd" Sep 10 12:29:11.651130 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:29:11.651115 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3a03fcba239265e913f999b9820e0c38a6d3d65fca2f3e2532b4d167fd0b9cd\": container with ID starting with d3a03fcba239265e913f999b9820e0c38a6d3d65fca2f3e2532b4d167fd0b9cd not found: ID does not exist" containerID="d3a03fcba239265e913f999b9820e0c38a6d3d65fca2f3e2532b4d167fd0b9cd" Sep 10 12:29:11.651182 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.651133 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3a03fcba239265e913f999b9820e0c38a6d3d65fca2f3e2532b4d167fd0b9cd"} err="failed to get container status \"d3a03fcba239265e913f999b9820e0c38a6d3d65fca2f3e2532b4d167fd0b9cd\": rpc error: code = NotFound desc = could not find container \"d3a03fcba239265e913f999b9820e0c38a6d3d65fca2f3e2532b4d167fd0b9cd\": container with ID starting with d3a03fcba239265e913f999b9820e0c38a6d3d65fca2f3e2532b4d167fd0b9cd not found: ID does not exist" Sep 10 12:29:11.651182 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.651144 2580 scope.go:117] "RemoveContainer" containerID="dc87ea858365977bfcf8ea10957a368e05e2fe969dfbadf57aa8b3ef8c395d76" Sep 10 12:29:11.654496 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.654472 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9"] Sep 10 12:29:11.658557 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.658540 2580 scope.go:117] "RemoveContainer" containerID="f2a1c93943977c8ab16d32cc23ebeae29b0ba06520b1bf97f4ad16b10793c5f3" Sep 10 12:29:11.659449 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.659427 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-67d6b6f8f7-b2sx9"] Sep 10 12:29:11.666180 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.666165 2580 scope.go:117] "RemoveContainer" containerID="be86f7ec3ce31061678962a95c92729fede788683b97b8e0fdfece51d6be0676" Sep 10 12:29:11.672893 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.672874 2580 scope.go:117] "RemoveContainer" containerID="dc781d101bc450900821fc6902d3e255222ece7253b0ab8eb37d413d4428729b" Sep 10 12:29:11.679952 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.679932 2580 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0570e6d2-af94-447b-a8e2-73b63855d2ee-ui-tls\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:29:11.679952 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.679951 2580 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0570e6d2-af94-447b-a8e2-73b63855d2ee-online-tls\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:29:11.680089 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.679959 2580 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0570e6d2-af94-447b-a8e2-73b63855d2ee-feast-data\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:29:11.680089 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.679971 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mqv7h\" (UniqueName: \"kubernetes.io/projected/0570e6d2-af94-447b-a8e2-73b63855d2ee-kube-api-access-mqv7h\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:29:11.680089 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.679981 2580 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0570e6d2-af94-447b-a8e2-73b63855d2ee-offline-tls\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:29:11.680089 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.679995 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0570e6d2-af94-447b-a8e2-73b63855d2ee-registry-tls\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:29:11.681302 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.681290 2580 scope.go:117] "RemoveContainer" containerID="1759c913c67cccf2075f0c07ec5a82c3c4a2975231626df0e01c0c7e59766506" Sep 10 12:29:11.688106 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.688090 2580 scope.go:117] "RemoveContainer" containerID="5397a6886e74ce86939c9b7646517157514fe4a771766be23c94837b495e7be9" Sep 10 12:29:11.694894 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.694874 2580 scope.go:117] "RemoveContainer" containerID="dc87ea858365977bfcf8ea10957a368e05e2fe969dfbadf57aa8b3ef8c395d76" Sep 10 12:29:11.695144 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:29:11.695128 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc87ea858365977bfcf8ea10957a368e05e2fe969dfbadf57aa8b3ef8c395d76\": container with ID starting with dc87ea858365977bfcf8ea10957a368e05e2fe969dfbadf57aa8b3ef8c395d76 not found: ID does not exist" containerID="dc87ea858365977bfcf8ea10957a368e05e2fe969dfbadf57aa8b3ef8c395d76" Sep 10 12:29:11.695199 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.695153 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc87ea858365977bfcf8ea10957a368e05e2fe969dfbadf57aa8b3ef8c395d76"} err="failed to get container status \"dc87ea858365977bfcf8ea10957a368e05e2fe969dfbadf57aa8b3ef8c395d76\": rpc error: code = NotFound desc = could not find container \"dc87ea858365977bfcf8ea10957a368e05e2fe969dfbadf57aa8b3ef8c395d76\": container with ID starting with dc87ea858365977bfcf8ea10957a368e05e2fe969dfbadf57aa8b3ef8c395d76 not found: ID does not exist" Sep 10 12:29:11.695199 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.695172 2580 scope.go:117] "RemoveContainer" containerID="f2a1c93943977c8ab16d32cc23ebeae29b0ba06520b1bf97f4ad16b10793c5f3" Sep 10 12:29:11.695394 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:29:11.695377 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2a1c93943977c8ab16d32cc23ebeae29b0ba06520b1bf97f4ad16b10793c5f3\": container with ID starting with f2a1c93943977c8ab16d32cc23ebeae29b0ba06520b1bf97f4ad16b10793c5f3 not found: ID does not exist" containerID="f2a1c93943977c8ab16d32cc23ebeae29b0ba06520b1bf97f4ad16b10793c5f3" Sep 10 12:29:11.695456 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.695404 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2a1c93943977c8ab16d32cc23ebeae29b0ba06520b1bf97f4ad16b10793c5f3"} err="failed to get container status \"f2a1c93943977c8ab16d32cc23ebeae29b0ba06520b1bf97f4ad16b10793c5f3\": rpc error: code = NotFound desc = could not find container \"f2a1c93943977c8ab16d32cc23ebeae29b0ba06520b1bf97f4ad16b10793c5f3\": container with ID starting with f2a1c93943977c8ab16d32cc23ebeae29b0ba06520b1bf97f4ad16b10793c5f3 not found: ID does not exist" Sep 10 12:29:11.695456 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.695426 2580 scope.go:117] "RemoveContainer" containerID="be86f7ec3ce31061678962a95c92729fede788683b97b8e0fdfece51d6be0676" Sep 10 12:29:11.695653 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:29:11.695636 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be86f7ec3ce31061678962a95c92729fede788683b97b8e0fdfece51d6be0676\": container with ID starting with be86f7ec3ce31061678962a95c92729fede788683b97b8e0fdfece51d6be0676 not found: ID does not exist" containerID="be86f7ec3ce31061678962a95c92729fede788683b97b8e0fdfece51d6be0676" Sep 10 12:29:11.695687 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.695656 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be86f7ec3ce31061678962a95c92729fede788683b97b8e0fdfece51d6be0676"} err="failed to get container status \"be86f7ec3ce31061678962a95c92729fede788683b97b8e0fdfece51d6be0676\": rpc error: code = NotFound desc = could not find container \"be86f7ec3ce31061678962a95c92729fede788683b97b8e0fdfece51d6be0676\": container with ID starting with be86f7ec3ce31061678962a95c92729fede788683b97b8e0fdfece51d6be0676 not found: ID does not exist" Sep 10 12:29:11.695687 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.695668 2580 scope.go:117] "RemoveContainer" containerID="dc781d101bc450900821fc6902d3e255222ece7253b0ab8eb37d413d4428729b" Sep 10 12:29:11.695920 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:29:11.695898 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc781d101bc450900821fc6902d3e255222ece7253b0ab8eb37d413d4428729b\": container with ID starting with dc781d101bc450900821fc6902d3e255222ece7253b0ab8eb37d413d4428729b not found: ID does not exist" containerID="dc781d101bc450900821fc6902d3e255222ece7253b0ab8eb37d413d4428729b" Sep 10 12:29:11.696009 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.695921 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc781d101bc450900821fc6902d3e255222ece7253b0ab8eb37d413d4428729b"} err="failed to get container status \"dc781d101bc450900821fc6902d3e255222ece7253b0ab8eb37d413d4428729b\": rpc error: code = NotFound desc = could not find container \"dc781d101bc450900821fc6902d3e255222ece7253b0ab8eb37d413d4428729b\": container with ID starting with dc781d101bc450900821fc6902d3e255222ece7253b0ab8eb37d413d4428729b not found: ID does not exist" Sep 10 12:29:11.696009 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.695935 2580 scope.go:117] "RemoveContainer" containerID="1759c913c67cccf2075f0c07ec5a82c3c4a2975231626df0e01c0c7e59766506" Sep 10 12:29:11.696168 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:29:11.696150 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1759c913c67cccf2075f0c07ec5a82c3c4a2975231626df0e01c0c7e59766506\": container with ID starting with 1759c913c67cccf2075f0c07ec5a82c3c4a2975231626df0e01c0c7e59766506 not found: ID does not exist" containerID="1759c913c67cccf2075f0c07ec5a82c3c4a2975231626df0e01c0c7e59766506" Sep 10 12:29:11.696224 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.696174 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1759c913c67cccf2075f0c07ec5a82c3c4a2975231626df0e01c0c7e59766506"} err="failed to get container status \"1759c913c67cccf2075f0c07ec5a82c3c4a2975231626df0e01c0c7e59766506\": rpc error: code = NotFound desc = could not find container \"1759c913c67cccf2075f0c07ec5a82c3c4a2975231626df0e01c0c7e59766506\": container with ID starting with 1759c913c67cccf2075f0c07ec5a82c3c4a2975231626df0e01c0c7e59766506 not found: ID does not exist" Sep 10 12:29:11.696224 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.696194 2580 scope.go:117] "RemoveContainer" containerID="5397a6886e74ce86939c9b7646517157514fe4a771766be23c94837b495e7be9" Sep 10 12:29:11.696421 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:29:11.696406 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5397a6886e74ce86939c9b7646517157514fe4a771766be23c94837b495e7be9\": container with ID starting with 5397a6886e74ce86939c9b7646517157514fe4a771766be23c94837b495e7be9 not found: ID does not exist" containerID="5397a6886e74ce86939c9b7646517157514fe4a771766be23c94837b495e7be9" Sep 10 12:29:11.696464 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.696425 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5397a6886e74ce86939c9b7646517157514fe4a771766be23c94837b495e7be9"} err="failed to get container status \"5397a6886e74ce86939c9b7646517157514fe4a771766be23c94837b495e7be9\": rpc error: code = NotFound desc = could not find container \"5397a6886e74ce86939c9b7646517157514fe4a771766be23c94837b495e7be9\": container with ID starting with 5397a6886e74ce86939c9b7646517157514fe4a771766be23c94837b495e7be9 not found: ID does not exist" Sep 10 12:29:11.696464 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.696438 2580 scope.go:117] "RemoveContainer" containerID="dc87ea858365977bfcf8ea10957a368e05e2fe969dfbadf57aa8b3ef8c395d76" Sep 10 12:29:11.696675 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.696659 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc87ea858365977bfcf8ea10957a368e05e2fe969dfbadf57aa8b3ef8c395d76"} err="failed to get container status \"dc87ea858365977bfcf8ea10957a368e05e2fe969dfbadf57aa8b3ef8c395d76\": rpc error: code = NotFound desc = could not find container \"dc87ea858365977bfcf8ea10957a368e05e2fe969dfbadf57aa8b3ef8c395d76\": container with ID starting with dc87ea858365977bfcf8ea10957a368e05e2fe969dfbadf57aa8b3ef8c395d76 not found: ID does not exist" Sep 10 12:29:11.696717 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.696675 2580 scope.go:117] "RemoveContainer" containerID="f2a1c93943977c8ab16d32cc23ebeae29b0ba06520b1bf97f4ad16b10793c5f3" Sep 10 12:29:11.696913 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.696894 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2a1c93943977c8ab16d32cc23ebeae29b0ba06520b1bf97f4ad16b10793c5f3"} err="failed to get container status \"f2a1c93943977c8ab16d32cc23ebeae29b0ba06520b1bf97f4ad16b10793c5f3\": rpc error: code = NotFound desc = could not find container \"f2a1c93943977c8ab16d32cc23ebeae29b0ba06520b1bf97f4ad16b10793c5f3\": container with ID starting with f2a1c93943977c8ab16d32cc23ebeae29b0ba06520b1bf97f4ad16b10793c5f3 not found: ID does not exist" Sep 10 12:29:11.696913 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.696912 2580 scope.go:117] "RemoveContainer" containerID="be86f7ec3ce31061678962a95c92729fede788683b97b8e0fdfece51d6be0676" Sep 10 12:29:11.697122 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.697106 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be86f7ec3ce31061678962a95c92729fede788683b97b8e0fdfece51d6be0676"} err="failed to get container status \"be86f7ec3ce31061678962a95c92729fede788683b97b8e0fdfece51d6be0676\": rpc error: code = NotFound desc = could not find container \"be86f7ec3ce31061678962a95c92729fede788683b97b8e0fdfece51d6be0676\": container with ID starting with be86f7ec3ce31061678962a95c92729fede788683b97b8e0fdfece51d6be0676 not found: ID does not exist" Sep 10 12:29:11.697162 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.697122 2580 scope.go:117] "RemoveContainer" containerID="dc781d101bc450900821fc6902d3e255222ece7253b0ab8eb37d413d4428729b" Sep 10 12:29:11.697342 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.697323 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc781d101bc450900821fc6902d3e255222ece7253b0ab8eb37d413d4428729b"} err="failed to get container status \"dc781d101bc450900821fc6902d3e255222ece7253b0ab8eb37d413d4428729b\": rpc error: code = NotFound desc = could not find container \"dc781d101bc450900821fc6902d3e255222ece7253b0ab8eb37d413d4428729b\": container with ID starting with dc781d101bc450900821fc6902d3e255222ece7253b0ab8eb37d413d4428729b not found: ID does not exist" Sep 10 12:29:11.697392 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.697342 2580 scope.go:117] "RemoveContainer" containerID="1759c913c67cccf2075f0c07ec5a82c3c4a2975231626df0e01c0c7e59766506" Sep 10 12:29:11.697587 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.697571 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1759c913c67cccf2075f0c07ec5a82c3c4a2975231626df0e01c0c7e59766506"} err="failed to get container status \"1759c913c67cccf2075f0c07ec5a82c3c4a2975231626df0e01c0c7e59766506\": rpc error: code = NotFound desc = could not find container \"1759c913c67cccf2075f0c07ec5a82c3c4a2975231626df0e01c0c7e59766506\": container with ID starting with 1759c913c67cccf2075f0c07ec5a82c3c4a2975231626df0e01c0c7e59766506 not found: ID does not exist" Sep 10 12:29:11.697630 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.697588 2580 scope.go:117] "RemoveContainer" containerID="5397a6886e74ce86939c9b7646517157514fe4a771766be23c94837b495e7be9" Sep 10 12:29:11.697794 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:11.697778 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5397a6886e74ce86939c9b7646517157514fe4a771766be23c94837b495e7be9"} err="failed to get container status \"5397a6886e74ce86939c9b7646517157514fe4a771766be23c94837b495e7be9\": rpc error: code = NotFound desc = could not find container \"5397a6886e74ce86939c9b7646517157514fe4a771766be23c94837b495e7be9\": container with ID starting with 5397a6886e74ce86939c9b7646517157514fe4a771766be23c94837b495e7be9 not found: ID does not exist" Sep 10 12:29:12.556979 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:12.556947 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0570e6d2-af94-447b-a8e2-73b63855d2ee" path="/var/lib/kubelet/pods/0570e6d2-af94-447b-a8e2-73b63855d2ee/volumes" Sep 10 12:29:12.557562 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:12.557548 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="398fbfd2-4051-4c06-a844-06f4def11fd2" path="/var/lib/kubelet/pods/398fbfd2-4051-4c06-a844-06f4def11fd2/volumes" Sep 10 12:29:23.739820 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.739790 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/postgres-864846d4f5-sfwpd"] Sep 10 12:29:23.740295 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.740276 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerName="ui" Sep 10 12:29:23.740429 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.740299 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerName="ui" Sep 10 12:29:23.740429 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.740316 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerName="offline" Sep 10 12:29:23.740429 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.740324 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerName="offline" Sep 10 12:29:23.740429 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.740343 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="398fbfd2-4051-4c06-a844-06f4def11fd2" containerName="online" Sep 10 12:29:23.740429 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.740351 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="398fbfd2-4051-4c06-a844-06f4def11fd2" containerName="online" Sep 10 12:29:23.740429 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.740363 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="398fbfd2-4051-4c06-a844-06f4def11fd2" containerName="feast-apply" Sep 10 12:29:23.740429 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.740370 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="398fbfd2-4051-4c06-a844-06f4def11fd2" containerName="feast-apply" Sep 10 12:29:23.740429 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.740385 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerName="registry" Sep 10 12:29:23.740429 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.740393 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerName="registry" Sep 10 12:29:23.740429 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.740402 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="398fbfd2-4051-4c06-a844-06f4def11fd2" containerName="offline" Sep 10 12:29:23.740429 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.740409 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="398fbfd2-4051-4c06-a844-06f4def11fd2" containerName="offline" Sep 10 12:29:23.740429 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.740417 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="398fbfd2-4051-4c06-a844-06f4def11fd2" containerName="ui" Sep 10 12:29:23.740429 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.740426 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="398fbfd2-4051-4c06-a844-06f4def11fd2" containerName="ui" Sep 10 12:29:23.740429 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.740434 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="398fbfd2-4051-4c06-a844-06f4def11fd2" containerName="feast-init" Sep 10 12:29:23.741077 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.740442 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="398fbfd2-4051-4c06-a844-06f4def11fd2" containerName="feast-init" Sep 10 12:29:23.741077 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.740457 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerName="feast-init" Sep 10 12:29:23.741077 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.740465 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerName="feast-init" Sep 10 12:29:23.741077 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.740473 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerName="feast-apply" Sep 10 12:29:23.741077 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.740481 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerName="feast-apply" Sep 10 12:29:23.741077 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.740489 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerName="online" Sep 10 12:29:23.741077 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.740497 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerName="online" Sep 10 12:29:23.741077 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.740574 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerName="offline" Sep 10 12:29:23.741077 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.740586 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="398fbfd2-4051-4c06-a844-06f4def11fd2" containerName="ui" Sep 10 12:29:23.741077 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.740595 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="398fbfd2-4051-4c06-a844-06f4def11fd2" containerName="offline" Sep 10 12:29:23.741077 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.740608 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerName="online" Sep 10 12:29:23.741077 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.740616 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerName="ui" Sep 10 12:29:23.741077 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.740625 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="398fbfd2-4051-4c06-a844-06f4def11fd2" containerName="online" Sep 10 12:29:23.741077 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.740634 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0570e6d2-af94-447b-a8e2-73b63855d2ee" containerName="registry" Sep 10 12:29:23.746229 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.746210 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-sfwpd" Sep 10 12:29:23.749187 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.749009 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\"" Sep 10 12:29:23.749319 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.749214 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-h64cm\"" Sep 10 12:29:23.749905 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.749883 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 10 12:29:23.750379 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.750365 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 10 12:29:23.750841 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.750811 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-sfwpd"] Sep 10 12:29:23.777767 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.777734 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/5804a078-1b2a-44b1-8064-6636b90c6249-postgresdata\") pod \"postgres-864846d4f5-sfwpd\" (UID: \"5804a078-1b2a-44b1-8064-6636b90c6249\") " pod="test-ns-feast/postgres-864846d4f5-sfwpd" Sep 10 12:29:23.777907 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.777787 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wjwb\" (UniqueName: \"kubernetes.io/projected/5804a078-1b2a-44b1-8064-6636b90c6249-kube-api-access-5wjwb\") pod \"postgres-864846d4f5-sfwpd\" (UID: \"5804a078-1b2a-44b1-8064-6636b90c6249\") " pod="test-ns-feast/postgres-864846d4f5-sfwpd" Sep 10 12:29:23.878710 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.878670 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/5804a078-1b2a-44b1-8064-6636b90c6249-postgresdata\") pod \"postgres-864846d4f5-sfwpd\" (UID: \"5804a078-1b2a-44b1-8064-6636b90c6249\") " pod="test-ns-feast/postgres-864846d4f5-sfwpd" Sep 10 12:29:23.878710 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.878718 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wjwb\" (UniqueName: \"kubernetes.io/projected/5804a078-1b2a-44b1-8064-6636b90c6249-kube-api-access-5wjwb\") pod \"postgres-864846d4f5-sfwpd\" (UID: \"5804a078-1b2a-44b1-8064-6636b90c6249\") " pod="test-ns-feast/postgres-864846d4f5-sfwpd" Sep 10 12:29:23.879131 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.879108 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/5804a078-1b2a-44b1-8064-6636b90c6249-postgresdata\") pod \"postgres-864846d4f5-sfwpd\" (UID: \"5804a078-1b2a-44b1-8064-6636b90c6249\") " pod="test-ns-feast/postgres-864846d4f5-sfwpd" Sep 10 12:29:23.887968 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:23.887943 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wjwb\" (UniqueName: \"kubernetes.io/projected/5804a078-1b2a-44b1-8064-6636b90c6249-kube-api-access-5wjwb\") pod \"postgres-864846d4f5-sfwpd\" (UID: \"5804a078-1b2a-44b1-8064-6636b90c6249\") " pod="test-ns-feast/postgres-864846d4f5-sfwpd" Sep 10 12:29:24.065098 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:24.065020 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-sfwpd" Sep 10 12:29:24.331680 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:24.331603 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-sfwpd"] Sep 10 12:29:24.334331 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:29:24.334306 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5804a078_1b2a_44b1_8064_6636b90c6249.slice/crio-b925434fc9292d5063014512c89d92f0aff2e3a98ef9e63d0d5a6fbcb2e0a289 WatchSource:0}: Error finding container b925434fc9292d5063014512c89d92f0aff2e3a98ef9e63d0d5a6fbcb2e0a289: Status 404 returned error can't find the container with id b925434fc9292d5063014512c89d92f0aff2e3a98ef9e63d0d5a6fbcb2e0a289 Sep 10 12:29:24.336180 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:24.336168 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 12:29:24.671239 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:24.671210 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-sfwpd" event={"ID":"5804a078-1b2a-44b1-8064-6636b90c6249","Type":"ContainerStarted","Data":"b925434fc9292d5063014512c89d92f0aff2e3a98ef9e63d0d5a6fbcb2e0a289"} Sep 10 12:29:29.696022 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:29.695978 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-sfwpd" event={"ID":"5804a078-1b2a-44b1-8064-6636b90c6249","Type":"ContainerStarted","Data":"421239a9cc936d12a7538f53a45c9668332da2aece89f6748d970519564b8291"} Sep 10 12:29:29.712873 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:29.712816 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/postgres-864846d4f5-sfwpd" podStartSLOduration=1.687315116 podStartE2EDuration="6.712800949s" totalImagesPullingTime="5.025485833s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:29:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:29:29.711811296 +0000 UTC m=+689.719127678" watchObservedRunningTime="2026-09-10 12:29:29.712800949 +0000 UTC m=+689.720117330" Sep 10 12:29:32.567383 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:32.567349 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-credit-scoring-55756c5579-tsx4w"] Sep 10 12:29:32.571109 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:32.571086 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" Sep 10 12:29:32.573817 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:32.573797 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-registry-tls\"" Sep 10 12:29:32.573946 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:32.573795 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-offline-tls\"" Sep 10 12:29:32.573946 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:32.573882 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-online-tls\"" Sep 10 12:29:32.573946 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:32.573907 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-9pbhb\"" Sep 10 12:29:32.574074 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:32.573981 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Sep 10 12:29:32.581654 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:32.581633 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-55756c5579-tsx4w"] Sep 10 12:29:32.654694 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:32.654666 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/962bb336-b5a1-4231-9dda-d562e70431a7-feast-data\") pod \"feast-credit-scoring-55756c5579-tsx4w\" (UID: \"962bb336-b5a1-4231-9dda-d562e70431a7\") " pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" Sep 10 12:29:32.654694 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:32.654701 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnss6\" (UniqueName: \"kubernetes.io/projected/962bb336-b5a1-4231-9dda-d562e70431a7-kube-api-access-gnss6\") pod \"feast-credit-scoring-55756c5579-tsx4w\" (UID: \"962bb336-b5a1-4231-9dda-d562e70431a7\") " pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" Sep 10 12:29:32.654950 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:32.654730 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/962bb336-b5a1-4231-9dda-d562e70431a7-online-tls\") pod \"feast-credit-scoring-55756c5579-tsx4w\" (UID: \"962bb336-b5a1-4231-9dda-d562e70431a7\") " pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" Sep 10 12:29:32.654950 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:32.654749 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/962bb336-b5a1-4231-9dda-d562e70431a7-offline-tls\") pod \"feast-credit-scoring-55756c5579-tsx4w\" (UID: \"962bb336-b5a1-4231-9dda-d562e70431a7\") " pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" Sep 10 12:29:32.654950 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:32.654853 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/962bb336-b5a1-4231-9dda-d562e70431a7-registry-tls\") pod \"feast-credit-scoring-55756c5579-tsx4w\" (UID: \"962bb336-b5a1-4231-9dda-d562e70431a7\") " pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" Sep 10 12:29:32.756215 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:32.756176 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/962bb336-b5a1-4231-9dda-d562e70431a7-feast-data\") pod \"feast-credit-scoring-55756c5579-tsx4w\" (UID: \"962bb336-b5a1-4231-9dda-d562e70431a7\") " pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" Sep 10 12:29:32.756403 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:32.756248 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnss6\" (UniqueName: \"kubernetes.io/projected/962bb336-b5a1-4231-9dda-d562e70431a7-kube-api-access-gnss6\") pod \"feast-credit-scoring-55756c5579-tsx4w\" (UID: \"962bb336-b5a1-4231-9dda-d562e70431a7\") " pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" Sep 10 12:29:32.756403 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:32.756294 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/962bb336-b5a1-4231-9dda-d562e70431a7-online-tls\") pod \"feast-credit-scoring-55756c5579-tsx4w\" (UID: \"962bb336-b5a1-4231-9dda-d562e70431a7\") " pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" Sep 10 12:29:32.756403 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:32.756316 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/962bb336-b5a1-4231-9dda-d562e70431a7-offline-tls\") pod \"feast-credit-scoring-55756c5579-tsx4w\" (UID: \"962bb336-b5a1-4231-9dda-d562e70431a7\") " pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" Sep 10 12:29:32.756403 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:32.756366 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/962bb336-b5a1-4231-9dda-d562e70431a7-registry-tls\") pod \"feast-credit-scoring-55756c5579-tsx4w\" (UID: \"962bb336-b5a1-4231-9dda-d562e70431a7\") " pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" Sep 10 12:29:32.756588 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:29:32.756423 2580 secret.go:189] Couldn't get secret test-ns-feast/feast-credit-scoring-online-tls: secret "feast-credit-scoring-online-tls" not found Sep 10 12:29:32.756588 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:29:32.756468 2580 secret.go:189] Couldn't get secret test-ns-feast/feast-credit-scoring-registry-tls: secret "feast-credit-scoring-registry-tls" not found Sep 10 12:29:32.756588 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:29:32.756497 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/962bb336-b5a1-4231-9dda-d562e70431a7-online-tls podName:962bb336-b5a1-4231-9dda-d562e70431a7 nodeName:}" failed. No retries permitted until 2026-09-10 12:29:33.25648156 +0000 UTC m=+693.263797920 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/962bb336-b5a1-4231-9dda-d562e70431a7-online-tls") pod "feast-credit-scoring-55756c5579-tsx4w" (UID: "962bb336-b5a1-4231-9dda-d562e70431a7") : secret "feast-credit-scoring-online-tls" not found Sep 10 12:29:32.756588 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:29:32.756526 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/962bb336-b5a1-4231-9dda-d562e70431a7-registry-tls podName:962bb336-b5a1-4231-9dda-d562e70431a7 nodeName:}" failed. No retries permitted until 2026-09-10 12:29:33.256509958 +0000 UTC m=+693.263826318 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/962bb336-b5a1-4231-9dda-d562e70431a7-registry-tls") pod "feast-credit-scoring-55756c5579-tsx4w" (UID: "962bb336-b5a1-4231-9dda-d562e70431a7") : secret "feast-credit-scoring-registry-tls" not found Sep 10 12:29:32.756588 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:32.756583 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/962bb336-b5a1-4231-9dda-d562e70431a7-feast-data\") pod \"feast-credit-scoring-55756c5579-tsx4w\" (UID: \"962bb336-b5a1-4231-9dda-d562e70431a7\") " pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" Sep 10 12:29:32.758604 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:32.758587 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/962bb336-b5a1-4231-9dda-d562e70431a7-offline-tls\") pod \"feast-credit-scoring-55756c5579-tsx4w\" (UID: \"962bb336-b5a1-4231-9dda-d562e70431a7\") " pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" Sep 10 12:29:32.764772 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:32.764752 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnss6\" (UniqueName: \"kubernetes.io/projected/962bb336-b5a1-4231-9dda-d562e70431a7-kube-api-access-gnss6\") pod \"feast-credit-scoring-55756c5579-tsx4w\" (UID: \"962bb336-b5a1-4231-9dda-d562e70431a7\") " pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" Sep 10 12:29:33.261382 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:33.261326 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/962bb336-b5a1-4231-9dda-d562e70431a7-online-tls\") pod \"feast-credit-scoring-55756c5579-tsx4w\" (UID: \"962bb336-b5a1-4231-9dda-d562e70431a7\") " pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" Sep 10 12:29:33.261586 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:33.261413 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/962bb336-b5a1-4231-9dda-d562e70431a7-registry-tls\") pod \"feast-credit-scoring-55756c5579-tsx4w\" (UID: \"962bb336-b5a1-4231-9dda-d562e70431a7\") " pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" Sep 10 12:29:33.263778 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:33.263739 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/962bb336-b5a1-4231-9dda-d562e70431a7-registry-tls\") pod \"feast-credit-scoring-55756c5579-tsx4w\" (UID: \"962bb336-b5a1-4231-9dda-d562e70431a7\") " pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" Sep 10 12:29:33.263899 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:33.263804 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/962bb336-b5a1-4231-9dda-d562e70431a7-online-tls\") pod \"feast-credit-scoring-55756c5579-tsx4w\" (UID: \"962bb336-b5a1-4231-9dda-d562e70431a7\") " pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" Sep 10 12:29:33.479543 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:33.479510 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" Sep 10 12:29:33.756417 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:33.756390 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-55756c5579-tsx4w"] Sep 10 12:29:33.756880 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:29:33.756841 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod962bb336_b5a1_4231_9dda_d562e70431a7.slice/crio-f81143de92c9e03a700eaa684e57591de41564286805fa8deaaf8486cc1d75c2 WatchSource:0}: Error finding container f81143de92c9e03a700eaa684e57591de41564286805fa8deaaf8486cc1d75c2: Status 404 returned error can't find the container with id f81143de92c9e03a700eaa684e57591de41564286805fa8deaaf8486cc1d75c2 Sep 10 12:29:34.718586 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:34.718541 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" event={"ID":"962bb336-b5a1-4231-9dda-d562e70431a7","Type":"ContainerStarted","Data":"6ddd96ce11642e5c70c8119453efa6c239905de580b67e28d75479fdaf23b65e"} Sep 10 12:29:34.718756 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:34.718616 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" event={"ID":"962bb336-b5a1-4231-9dda-d562e70431a7","Type":"ContainerStarted","Data":"f81143de92c9e03a700eaa684e57591de41564286805fa8deaaf8486cc1d75c2"} Sep 10 12:29:35.724073 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:35.724039 2580 generic.go:356] "Generic (PLEG): container finished" podID="962bb336-b5a1-4231-9dda-d562e70431a7" containerID="6ddd96ce11642e5c70c8119453efa6c239905de580b67e28d75479fdaf23b65e" exitCode=0 Sep 10 12:29:35.724444 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:35.724129 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" event={"ID":"962bb336-b5a1-4231-9dda-d562e70431a7","Type":"ContainerDied","Data":"6ddd96ce11642e5c70c8119453efa6c239905de580b67e28d75479fdaf23b65e"} Sep 10 12:29:36.730744 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:36.730712 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" event={"ID":"962bb336-b5a1-4231-9dda-d562e70431a7","Type":"ContainerStarted","Data":"cd15d5c32325bd972125631b8f14f6a092cd429b85cfc00c69b04c2575c2d8f1"} Sep 10 12:29:42.758949 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:42.758915 2580 generic.go:356] "Generic (PLEG): container finished" podID="962bb336-b5a1-4231-9dda-d562e70431a7" containerID="cd15d5c32325bd972125631b8f14f6a092cd429b85cfc00c69b04c2575c2d8f1" exitCode=0 Sep 10 12:29:42.759338 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:42.758982 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" event={"ID":"962bb336-b5a1-4231-9dda-d562e70431a7","Type":"ContainerDied","Data":"cd15d5c32325bd972125631b8f14f6a092cd429b85cfc00c69b04c2575c2d8f1"} Sep 10 12:29:43.767451 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:43.767412 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" event={"ID":"962bb336-b5a1-4231-9dda-d562e70431a7","Type":"ContainerStarted","Data":"a000eff6fe7ed7c7256c19da66f747d971412ba98a5701023938a713047cdb35"} Sep 10 12:29:43.767839 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:43.767485 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" event={"ID":"962bb336-b5a1-4231-9dda-d562e70431a7","Type":"ContainerStarted","Data":"b72c0b49aa444ad2e6454af1d66e9d81cc2d83de3550287609793484b5ff953f"} Sep 10 12:29:43.767839 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:43.767514 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" event={"ID":"962bb336-b5a1-4231-9dda-d562e70431a7","Type":"ContainerStarted","Data":"6f79c72b14ff4f914cdf05eb9fa0e7da05bc6ad77e61913649c2095e9ca500c5"} Sep 10 12:29:43.791665 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:43.791603 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" podStartSLOduration=11.791580873000001 podStartE2EDuration="11.791580873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:29:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:29:43.791229047 +0000 UTC m=+703.798545429" watchObservedRunningTime="2026-09-10 12:29:43.791580873 +0000 UTC m=+703.798897255" Sep 10 12:29:45.479962 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:45.479927 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" Sep 10 12:29:45.480528 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:45.480483 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" Sep 10 12:29:45.480528 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:45.480523 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" Sep 10 12:29:45.482048 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:45.482008 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" podUID="962bb336-b5a1-4231-9dda-d562e70431a7" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.24:6571: connect: connection refused" Sep 10 12:29:45.482164 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:45.482037 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" podUID="962bb336-b5a1-4231-9dda-d562e70431a7" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.24:8816: connect: connection refused" Sep 10 12:29:45.482164 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:45.482013 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" podUID="962bb336-b5a1-4231-9dda-d562e70431a7" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused" Sep 10 12:29:48.480120 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:48.480057 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" podUID="962bb336-b5a1-4231-9dda-d562e70431a7" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.24:6571: connect: connection refused" Sep 10 12:29:48.480120 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:48.480074 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" podUID="962bb336-b5a1-4231-9dda-d562e70431a7" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.24:8816: connect: connection refused" Sep 10 12:29:48.480639 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:48.480108 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" podUID="962bb336-b5a1-4231-9dda-d562e70431a7" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused" Sep 10 12:29:51.480766 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:51.480717 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" podUID="962bb336-b5a1-4231-9dda-d562e70431a7" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.24:6571: connect: connection refused" Sep 10 12:29:51.481757 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:51.481730 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" Sep 10 12:29:51.482228 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:51.482204 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" Sep 10 12:29:51.482654 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:51.482633 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" Sep 10 12:29:51.488828 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:51.488807 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" Sep 10 12:29:51.820212 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:51.820134 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" Sep 10 12:29:51.823557 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:51.823535 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" Sep 10 12:29:53.480756 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:53.480716 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" Sep 10 12:29:54.480472 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:54.480429 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" podUID="962bb336-b5a1-4231-9dda-d562e70431a7" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.24:6571: connect: connection refused" Sep 10 12:29:57.481113 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:57.481084 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" Sep 10 12:29:57.481821 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:29:57.481801 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" Sep 10 12:31:00.671289 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:00.671255 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-55756c5579-tsx4w"] Sep 10 12:31:00.671892 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:00.671535 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" podUID="962bb336-b5a1-4231-9dda-d562e70431a7" containerName="registry" containerID="cri-o://6f79c72b14ff4f914cdf05eb9fa0e7da05bc6ad77e61913649c2095e9ca500c5" gracePeriod=30 Sep 10 12:31:00.671892 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:00.671622 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" podUID="962bb336-b5a1-4231-9dda-d562e70431a7" containerName="online" containerID="cri-o://b72c0b49aa444ad2e6454af1d66e9d81cc2d83de3550287609793484b5ff953f" gracePeriod=30 Sep 10 12:31:00.671892 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:00.671622 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" podUID="962bb336-b5a1-4231-9dda-d562e70431a7" containerName="offline" containerID="cri-o://a000eff6fe7ed7c7256c19da66f747d971412ba98a5701023938a713047cdb35" gracePeriod=30 Sep 10 12:31:00.687271 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:00.687247 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-sfwpd"] Sep 10 12:31:00.687407 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:00.687390 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/postgres-864846d4f5-sfwpd" podUID="5804a078-1b2a-44b1-8064-6636b90c6249" containerName="postgres" containerID="cri-o://421239a9cc936d12a7538f53a45c9668332da2aece89f6748d970519564b8291" gracePeriod=30 Sep 10 12:31:00.921850 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:00.921792 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-sfwpd" Sep 10 12:31:00.924649 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:00.924631 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5wjwb\" (UniqueName: \"kubernetes.io/projected/5804a078-1b2a-44b1-8064-6636b90c6249-kube-api-access-5wjwb\") pod \"5804a078-1b2a-44b1-8064-6636b90c6249\" (UID: \"5804a078-1b2a-44b1-8064-6636b90c6249\") " Sep 10 12:31:00.924720 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:00.924664 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/5804a078-1b2a-44b1-8064-6636b90c6249-postgresdata\") pod \"5804a078-1b2a-44b1-8064-6636b90c6249\" (UID: \"5804a078-1b2a-44b1-8064-6636b90c6249\") " Sep 10 12:31:00.926566 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:00.926540 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5804a078-1b2a-44b1-8064-6636b90c6249-kube-api-access-5wjwb" (OuterVolumeSpecName: "kube-api-access-5wjwb") pod "5804a078-1b2a-44b1-8064-6636b90c6249" (UID: "5804a078-1b2a-44b1-8064-6636b90c6249"). InnerVolumeSpecName "kube-api-access-5wjwb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:31:00.955515 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:00.955486 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5804a078-1b2a-44b1-8064-6636b90c6249-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "5804a078-1b2a-44b1-8064-6636b90c6249" (UID: "5804a078-1b2a-44b1-8064-6636b90c6249"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 12:31:01.025777 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:01.025736 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wjwb\" (UniqueName: \"kubernetes.io/projected/5804a078-1b2a-44b1-8064-6636b90c6249-kube-api-access-5wjwb\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:31:01.025777 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:01.025761 2580 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/5804a078-1b2a-44b1-8064-6636b90c6249-postgresdata\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:31:01.110633 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:01.110601 2580 generic.go:356] "Generic (PLEG): container finished" podID="5804a078-1b2a-44b1-8064-6636b90c6249" containerID="421239a9cc936d12a7538f53a45c9668332da2aece89f6748d970519564b8291" exitCode=0 Sep 10 12:31:01.110838 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:01.110645 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-sfwpd" Sep 10 12:31:01.110838 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:01.110683 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-sfwpd" event={"ID":"5804a078-1b2a-44b1-8064-6636b90c6249","Type":"ContainerDied","Data":"421239a9cc936d12a7538f53a45c9668332da2aece89f6748d970519564b8291"} Sep 10 12:31:01.110838 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:01.110728 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-sfwpd" event={"ID":"5804a078-1b2a-44b1-8064-6636b90c6249","Type":"ContainerDied","Data":"b925434fc9292d5063014512c89d92f0aff2e3a98ef9e63d0d5a6fbcb2e0a289"} Sep 10 12:31:01.110838 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:01.110748 2580 scope.go:117] "RemoveContainer" containerID="421239a9cc936d12a7538f53a45c9668332da2aece89f6748d970519564b8291" Sep 10 12:31:01.119375 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:01.119355 2580 scope.go:117] "RemoveContainer" containerID="421239a9cc936d12a7538f53a45c9668332da2aece89f6748d970519564b8291" Sep 10 12:31:01.119647 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:31:01.119631 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"421239a9cc936d12a7538f53a45c9668332da2aece89f6748d970519564b8291\": container with ID starting with 421239a9cc936d12a7538f53a45c9668332da2aece89f6748d970519564b8291 not found: ID does not exist" containerID="421239a9cc936d12a7538f53a45c9668332da2aece89f6748d970519564b8291" Sep 10 12:31:01.119703 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:01.119660 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"421239a9cc936d12a7538f53a45c9668332da2aece89f6748d970519564b8291"} err="failed to get container status \"421239a9cc936d12a7538f53a45c9668332da2aece89f6748d970519564b8291\": rpc error: code = NotFound desc = could not find container \"421239a9cc936d12a7538f53a45c9668332da2aece89f6748d970519564b8291\": container with ID starting with 421239a9cc936d12a7538f53a45c9668332da2aece89f6748d970519564b8291 not found: ID does not exist" Sep 10 12:31:01.131726 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:01.131701 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-sfwpd"] Sep 10 12:31:01.135969 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:01.135946 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/postgres-864846d4f5-sfwpd"] Sep 10 12:31:01.820319 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:01.820226 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" podUID="962bb336-b5a1-4231-9dda-d562e70431a7" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused" Sep 10 12:31:02.118721 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:02.118642 2580 generic.go:356] "Generic (PLEG): container finished" podID="962bb336-b5a1-4231-9dda-d562e70431a7" containerID="b72c0b49aa444ad2e6454af1d66e9d81cc2d83de3550287609793484b5ff953f" exitCode=0 Sep 10 12:31:02.118721 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:02.118678 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" event={"ID":"962bb336-b5a1-4231-9dda-d562e70431a7","Type":"ContainerDied","Data":"b72c0b49aa444ad2e6454af1d66e9d81cc2d83de3550287609793484b5ff953f"} Sep 10 12:31:02.557089 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:02.557058 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5804a078-1b2a-44b1-8064-6636b90c6249" path="/var/lib/kubelet/pods/5804a078-1b2a-44b1-8064-6636b90c6249/volumes" Sep 10 12:31:11.821062 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:11.821022 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" podUID="962bb336-b5a1-4231-9dda-d562e70431a7" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused" Sep 10 12:31:21.820900 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:21.820829 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" podUID="962bb336-b5a1-4231-9dda-d562e70431a7" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused" Sep 10 12:31:21.821308 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:21.821004 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" Sep 10 12:31:31.239911 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:31.239881 2580 generic.go:356] "Generic (PLEG): container finished" podID="962bb336-b5a1-4231-9dda-d562e70431a7" containerID="a000eff6fe7ed7c7256c19da66f747d971412ba98a5701023938a713047cdb35" exitCode=137 Sep 10 12:31:31.239911 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:31.239906 2580 generic.go:356] "Generic (PLEG): container finished" podID="962bb336-b5a1-4231-9dda-d562e70431a7" containerID="6f79c72b14ff4f914cdf05eb9fa0e7da05bc6ad77e61913649c2095e9ca500c5" exitCode=137 Sep 10 12:31:31.240357 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:31.239931 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" event={"ID":"962bb336-b5a1-4231-9dda-d562e70431a7","Type":"ContainerDied","Data":"a000eff6fe7ed7c7256c19da66f747d971412ba98a5701023938a713047cdb35"} Sep 10 12:31:31.240357 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:31.239970 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" event={"ID":"962bb336-b5a1-4231-9dda-d562e70431a7","Type":"ContainerDied","Data":"6f79c72b14ff4f914cdf05eb9fa0e7da05bc6ad77e61913649c2095e9ca500c5"} Sep 10 12:31:31.321662 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:31.321638 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" Sep 10 12:31:31.482885 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:31.482785 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/962bb336-b5a1-4231-9dda-d562e70431a7-online-tls\") pod \"962bb336-b5a1-4231-9dda-d562e70431a7\" (UID: \"962bb336-b5a1-4231-9dda-d562e70431a7\") " Sep 10 12:31:31.482885 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:31.482852 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/962bb336-b5a1-4231-9dda-d562e70431a7-registry-tls\") pod \"962bb336-b5a1-4231-9dda-d562e70431a7\" (UID: \"962bb336-b5a1-4231-9dda-d562e70431a7\") " Sep 10 12:31:31.483173 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:31.482897 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/962bb336-b5a1-4231-9dda-d562e70431a7-offline-tls\") pod \"962bb336-b5a1-4231-9dda-d562e70431a7\" (UID: \"962bb336-b5a1-4231-9dda-d562e70431a7\") " Sep 10 12:31:31.483173 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:31.482943 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gnss6\" (UniqueName: \"kubernetes.io/projected/962bb336-b5a1-4231-9dda-d562e70431a7-kube-api-access-gnss6\") pod \"962bb336-b5a1-4231-9dda-d562e70431a7\" (UID: \"962bb336-b5a1-4231-9dda-d562e70431a7\") " Sep 10 12:31:31.483173 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:31.482965 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/962bb336-b5a1-4231-9dda-d562e70431a7-feast-data\") pod \"962bb336-b5a1-4231-9dda-d562e70431a7\" (UID: \"962bb336-b5a1-4231-9dda-d562e70431a7\") " Sep 10 12:31:31.485082 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:31.485049 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/962bb336-b5a1-4231-9dda-d562e70431a7-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "962bb336-b5a1-4231-9dda-d562e70431a7" (UID: "962bb336-b5a1-4231-9dda-d562e70431a7"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:31:31.485082 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:31.485071 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/962bb336-b5a1-4231-9dda-d562e70431a7-kube-api-access-gnss6" (OuterVolumeSpecName: "kube-api-access-gnss6") pod "962bb336-b5a1-4231-9dda-d562e70431a7" (UID: "962bb336-b5a1-4231-9dda-d562e70431a7"). InnerVolumeSpecName "kube-api-access-gnss6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:31:31.485298 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:31.485279 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/962bb336-b5a1-4231-9dda-d562e70431a7-online-tls" (OuterVolumeSpecName: "online-tls") pod "962bb336-b5a1-4231-9dda-d562e70431a7" (UID: "962bb336-b5a1-4231-9dda-d562e70431a7"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:31:31.485357 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:31.485309 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/962bb336-b5a1-4231-9dda-d562e70431a7-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "962bb336-b5a1-4231-9dda-d562e70431a7" (UID: "962bb336-b5a1-4231-9dda-d562e70431a7"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:31:31.490944 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:31.490920 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/962bb336-b5a1-4231-9dda-d562e70431a7-feast-data" (OuterVolumeSpecName: "feast-data") pod "962bb336-b5a1-4231-9dda-d562e70431a7" (UID: "962bb336-b5a1-4231-9dda-d562e70431a7"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 12:31:31.584615 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:31.584580 2580 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/962bb336-b5a1-4231-9dda-d562e70431a7-online-tls\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:31:31.584615 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:31.584609 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/962bb336-b5a1-4231-9dda-d562e70431a7-registry-tls\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:31:31.584615 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:31.584617 2580 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/962bb336-b5a1-4231-9dda-d562e70431a7-offline-tls\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:31:31.584615 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:31.584625 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gnss6\" (UniqueName: \"kubernetes.io/projected/962bb336-b5a1-4231-9dda-d562e70431a7-kube-api-access-gnss6\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:31:31.584887 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:31.584634 2580 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/962bb336-b5a1-4231-9dda-d562e70431a7-feast-data\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:31:32.246239 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:32.246204 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" event={"ID":"962bb336-b5a1-4231-9dda-d562e70431a7","Type":"ContainerDied","Data":"f81143de92c9e03a700eaa684e57591de41564286805fa8deaaf8486cc1d75c2"} Sep 10 12:31:32.246660 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:32.246278 2580 scope.go:117] "RemoveContainer" containerID="a000eff6fe7ed7c7256c19da66f747d971412ba98a5701023938a713047cdb35" Sep 10 12:31:32.246660 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:32.246216 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-55756c5579-tsx4w" Sep 10 12:31:32.255583 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:32.255564 2580 scope.go:117] "RemoveContainer" containerID="b72c0b49aa444ad2e6454af1d66e9d81cc2d83de3550287609793484b5ff953f" Sep 10 12:31:32.262816 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:32.262794 2580 scope.go:117] "RemoveContainer" containerID="6f79c72b14ff4f914cdf05eb9fa0e7da05bc6ad77e61913649c2095e9ca500c5" Sep 10 12:31:32.268037 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:32.268014 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-55756c5579-tsx4w"] Sep 10 12:31:32.272200 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:32.272177 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-credit-scoring-55756c5579-tsx4w"] Sep 10 12:31:32.273271 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:32.273256 2580 scope.go:117] "RemoveContainer" containerID="cd15d5c32325bd972125631b8f14f6a092cd429b85cfc00c69b04c2575c2d8f1" Sep 10 12:31:32.280112 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:32.280097 2580 scope.go:117] "RemoveContainer" containerID="6ddd96ce11642e5c70c8119453efa6c239905de580b67e28d75479fdaf23b65e" Sep 10 12:31:32.556409 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:31:32.556333 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="962bb336-b5a1-4231-9dda-d562e70431a7" path="/var/lib/kubelet/pods/962bb336-b5a1-4231-9dda-d562e70431a7/volumes" Sep 10 12:33:00.476299 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:00.476198 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tfcmz_e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58/ovn-acl-logging/0.log" Sep 10 12:33:00.477997 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:00.477976 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tfcmz_e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58/ovn-acl-logging/0.log" Sep 10 12:33:34.041492 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:34.041463 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-jpxt9"] Sep 10 12:33:34.042002 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:34.041798 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="962bb336-b5a1-4231-9dda-d562e70431a7" containerName="online" Sep 10 12:33:34.042002 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:34.041811 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="962bb336-b5a1-4231-9dda-d562e70431a7" containerName="online" Sep 10 12:33:34.042002 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:34.041819 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5804a078-1b2a-44b1-8064-6636b90c6249" containerName="postgres" Sep 10 12:33:34.042002 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:34.041824 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5804a078-1b2a-44b1-8064-6636b90c6249" containerName="postgres" Sep 10 12:33:34.042002 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:34.041842 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="962bb336-b5a1-4231-9dda-d562e70431a7" containerName="offline" Sep 10 12:33:34.042002 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:34.041848 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="962bb336-b5a1-4231-9dda-d562e70431a7" containerName="offline" Sep 10 12:33:34.042002 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:34.041878 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="962bb336-b5a1-4231-9dda-d562e70431a7" containerName="feast-init" Sep 10 12:33:34.042002 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:34.041883 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="962bb336-b5a1-4231-9dda-d562e70431a7" containerName="feast-init" Sep 10 12:33:34.042002 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:34.041892 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="962bb336-b5a1-4231-9dda-d562e70431a7" containerName="registry" Sep 10 12:33:34.042002 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:34.041898 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="962bb336-b5a1-4231-9dda-d562e70431a7" containerName="registry" Sep 10 12:33:34.042002 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:34.041913 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="962bb336-b5a1-4231-9dda-d562e70431a7" containerName="feast-apply" Sep 10 12:33:34.042002 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:34.041921 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="962bb336-b5a1-4231-9dda-d562e70431a7" containerName="feast-apply" Sep 10 12:33:34.042002 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:34.041975 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5804a078-1b2a-44b1-8064-6636b90c6249" containerName="postgres" Sep 10 12:33:34.042002 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:34.041985 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="962bb336-b5a1-4231-9dda-d562e70431a7" containerName="registry" Sep 10 12:33:34.042002 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:34.041995 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="962bb336-b5a1-4231-9dda-d562e70431a7" containerName="online" Sep 10 12:33:34.042002 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:34.042003 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="962bb336-b5a1-4231-9dda-d562e70431a7" containerName="offline" Sep 10 12:33:34.044876 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:34.044850 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-jpxt9" Sep 10 12:33:34.047581 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:34.047561 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\"" Sep 10 12:33:34.047718 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:34.047699 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-px7dj\"" Sep 10 12:33:34.047800 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:34.047722 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\"" Sep 10 12:33:34.057700 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:34.057676 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-jpxt9"] Sep 10 12:33:34.123011 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:34.122975 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bhgj\" (UniqueName: \"kubernetes.io/projected/07d57c47-d5b0-43b5-b54e-5d0b7325ae7e-kube-api-access-8bhgj\") pod \"redis-69bbd798b6-jpxt9\" (UID: \"07d57c47-d5b0-43b5-b54e-5d0b7325ae7e\") " pod="test-ns-feast-rest/redis-69bbd798b6-jpxt9" Sep 10 12:33:34.223345 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:34.223304 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bhgj\" (UniqueName: \"kubernetes.io/projected/07d57c47-d5b0-43b5-b54e-5d0b7325ae7e-kube-api-access-8bhgj\") pod \"redis-69bbd798b6-jpxt9\" (UID: \"07d57c47-d5b0-43b5-b54e-5d0b7325ae7e\") " pod="test-ns-feast-rest/redis-69bbd798b6-jpxt9" Sep 10 12:33:34.232216 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:34.232197 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bhgj\" (UniqueName: \"kubernetes.io/projected/07d57c47-d5b0-43b5-b54e-5d0b7325ae7e-kube-api-access-8bhgj\") pod \"redis-69bbd798b6-jpxt9\" (UID: \"07d57c47-d5b0-43b5-b54e-5d0b7325ae7e\") " pod="test-ns-feast-rest/redis-69bbd798b6-jpxt9" Sep 10 12:33:34.359724 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:34.359637 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-jpxt9" Sep 10 12:33:34.627067 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:34.626996 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-jpxt9"] Sep 10 12:33:34.630715 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:33:34.630690 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07d57c47_d5b0_43b5_b54e_5d0b7325ae7e.slice/crio-3eaf094ad2f41c538aa322088b6d673a1b9e583d6f8e23b96b51ef831c27d207 WatchSource:0}: Error finding container 3eaf094ad2f41c538aa322088b6d673a1b9e583d6f8e23b96b51ef831c27d207: Status 404 returned error can't find the container with id 3eaf094ad2f41c538aa322088b6d673a1b9e583d6f8e23b96b51ef831c27d207 Sep 10 12:33:34.711261 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:34.711233 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-jpxt9" event={"ID":"07d57c47-d5b0-43b5-b54e-5d0b7325ae7e","Type":"ContainerStarted","Data":"3eaf094ad2f41c538aa322088b6d673a1b9e583d6f8e23b96b51ef831c27d207"} Sep 10 12:33:36.725417 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:36.725376 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-jpxt9" event={"ID":"07d57c47-d5b0-43b5-b54e-5d0b7325ae7e","Type":"ContainerStarted","Data":"2c74d70a24bd0ae85c6dc7cd9517c30d6c0a841bd3f1d8fbd6dda01238a6698a"} Sep 10 12:33:36.741515 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:36.741466 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/redis-69bbd798b6-jpxt9" podStartSLOduration=1.509767703 podStartE2EDuration="2.741450588s" totalImagesPullingTime="1.231682885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:33:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:33:36.740375764 +0000 UTC m=+936.747692145" watchObservedRunningTime="2026-09-10 12:33:36.741450588 +0000 UTC m=+936.748766989" Sep 10 12:33:39.486474 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:39.486434 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-5ndhg"] Sep 10 12:33:39.490496 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:39.490471 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-5ndhg" Sep 10 12:33:39.493780 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:39.493754 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\"" Sep 10 12:33:39.494931 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:39.494906 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-5ndhg"] Sep 10 12:33:39.566625 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:39.566581 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/5bd89a92-2371-4743-94ef-2907d0cc3d76-postgresdata\") pod \"postgres-864846d4f5-5ndhg\" (UID: \"5bd89a92-2371-4743-94ef-2907d0cc3d76\") " pod="test-ns-feast-rest/postgres-864846d4f5-5ndhg" Sep 10 12:33:39.566808 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:39.566630 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v6ln\" (UniqueName: \"kubernetes.io/projected/5bd89a92-2371-4743-94ef-2907d0cc3d76-kube-api-access-6v6ln\") pod \"postgres-864846d4f5-5ndhg\" (UID: \"5bd89a92-2371-4743-94ef-2907d0cc3d76\") " pod="test-ns-feast-rest/postgres-864846d4f5-5ndhg" Sep 10 12:33:39.667625 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:39.667587 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/5bd89a92-2371-4743-94ef-2907d0cc3d76-postgresdata\") pod \"postgres-864846d4f5-5ndhg\" (UID: \"5bd89a92-2371-4743-94ef-2907d0cc3d76\") " pod="test-ns-feast-rest/postgres-864846d4f5-5ndhg" Sep 10 12:33:39.667625 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:39.667632 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v6ln\" (UniqueName: \"kubernetes.io/projected/5bd89a92-2371-4743-94ef-2907d0cc3d76-kube-api-access-6v6ln\") pod \"postgres-864846d4f5-5ndhg\" (UID: \"5bd89a92-2371-4743-94ef-2907d0cc3d76\") " pod="test-ns-feast-rest/postgres-864846d4f5-5ndhg" Sep 10 12:33:39.668094 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:39.668072 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/5bd89a92-2371-4743-94ef-2907d0cc3d76-postgresdata\") pod \"postgres-864846d4f5-5ndhg\" (UID: \"5bd89a92-2371-4743-94ef-2907d0cc3d76\") " pod="test-ns-feast-rest/postgres-864846d4f5-5ndhg" Sep 10 12:33:39.676525 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:39.676504 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v6ln\" (UniqueName: \"kubernetes.io/projected/5bd89a92-2371-4743-94ef-2907d0cc3d76-kube-api-access-6v6ln\") pod \"postgres-864846d4f5-5ndhg\" (UID: \"5bd89a92-2371-4743-94ef-2907d0cc3d76\") " pod="test-ns-feast-rest/postgres-864846d4f5-5ndhg" Sep 10 12:33:39.800940 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:39.800842 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-5ndhg" Sep 10 12:33:40.068773 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:40.068665 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-5ndhg"] Sep 10 12:33:40.072368 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:33:40.072337 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5bd89a92_2371_4743_94ef_2907d0cc3d76.slice/crio-e624a403f3eb74c02d8622f278c3fc81f9885c29ef845593969bb329f95fb1d0 WatchSource:0}: Error finding container e624a403f3eb74c02d8622f278c3fc81f9885c29ef845593969bb329f95fb1d0: Status 404 returned error can't find the container with id e624a403f3eb74c02d8622f278c3fc81f9885c29ef845593969bb329f95fb1d0 Sep 10 12:33:40.742153 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:40.742122 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-5ndhg" event={"ID":"5bd89a92-2371-4743-94ef-2907d0cc3d76","Type":"ContainerStarted","Data":"2b4e8ceba3d48012807be08a01188f8e3ffa31cddfbfee6a8426a74898d74b38"} Sep 10 12:33:40.742153 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:40.742160 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-5ndhg" event={"ID":"5bd89a92-2371-4743-94ef-2907d0cc3d76","Type":"ContainerStarted","Data":"e624a403f3eb74c02d8622f278c3fc81f9885c29ef845593969bb329f95fb1d0"} Sep 10 12:33:40.757950 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:40.757819 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/postgres-864846d4f5-5ndhg" podStartSLOduration=1.7578036369999999 podStartE2EDuration="1.757803637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:33:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:33:40.757192925 +0000 UTC m=+940.764509308" watchObservedRunningTime="2026-09-10 12:33:40.757803637 +0000 UTC m=+940.765120048" Sep 10 12:33:45.084342 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:45.084309 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k"] Sep 10 12:33:45.090527 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:45.090507 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" Sep 10 12:33:45.093513 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:45.093489 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-online-tls\"" Sep 10 12:33:45.093607 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:45.093517 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\"" Sep 10 12:33:45.093607 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:45.093531 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-dockercfg-lk6gg\"" Sep 10 12:33:45.093607 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:45.093497 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-registry-tls\"" Sep 10 12:33:45.101077 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:45.101054 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k"] Sep 10 12:33:45.218217 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:45.218182 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b7a108cc-ba0b-489e-ad11-7ee328edd6da-registry-tls\") pod \"feast-credit-scoring-747c6fb677-4h24k\" (UID: \"b7a108cc-ba0b-489e-ad11-7ee328edd6da\") " pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" Sep 10 12:33:45.218372 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:45.218227 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b7a108cc-ba0b-489e-ad11-7ee328edd6da-online-tls\") pod \"feast-credit-scoring-747c6fb677-4h24k\" (UID: \"b7a108cc-ba0b-489e-ad11-7ee328edd6da\") " pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" Sep 10 12:33:45.218372 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:45.218243 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b7a108cc-ba0b-489e-ad11-7ee328edd6da-feast-data\") pod \"feast-credit-scoring-747c6fb677-4h24k\" (UID: \"b7a108cc-ba0b-489e-ad11-7ee328edd6da\") " pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" Sep 10 12:33:45.218372 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:45.218322 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6fzt\" (UniqueName: \"kubernetes.io/projected/b7a108cc-ba0b-489e-ad11-7ee328edd6da-kube-api-access-t6fzt\") pod \"feast-credit-scoring-747c6fb677-4h24k\" (UID: \"b7a108cc-ba0b-489e-ad11-7ee328edd6da\") " pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" Sep 10 12:33:45.318875 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:45.318830 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b7a108cc-ba0b-489e-ad11-7ee328edd6da-online-tls\") pod \"feast-credit-scoring-747c6fb677-4h24k\" (UID: \"b7a108cc-ba0b-489e-ad11-7ee328edd6da\") " pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" Sep 10 12:33:45.318875 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:45.318877 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b7a108cc-ba0b-489e-ad11-7ee328edd6da-feast-data\") pod \"feast-credit-scoring-747c6fb677-4h24k\" (UID: \"b7a108cc-ba0b-489e-ad11-7ee328edd6da\") " pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" Sep 10 12:33:45.319082 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:45.318901 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6fzt\" (UniqueName: \"kubernetes.io/projected/b7a108cc-ba0b-489e-ad11-7ee328edd6da-kube-api-access-t6fzt\") pod \"feast-credit-scoring-747c6fb677-4h24k\" (UID: \"b7a108cc-ba0b-489e-ad11-7ee328edd6da\") " pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" Sep 10 12:33:45.319082 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:45.318970 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b7a108cc-ba0b-489e-ad11-7ee328edd6da-registry-tls\") pod \"feast-credit-scoring-747c6fb677-4h24k\" (UID: \"b7a108cc-ba0b-489e-ad11-7ee328edd6da\") " pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" Sep 10 12:33:45.319082 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:33:45.319055 2580 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-credit-scoring-registry-tls: secret "feast-credit-scoring-registry-tls" not found Sep 10 12:33:45.319197 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:33:45.319106 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7a108cc-ba0b-489e-ad11-7ee328edd6da-registry-tls podName:b7a108cc-ba0b-489e-ad11-7ee328edd6da nodeName:}" failed. No retries permitted until 2026-09-10 12:33:45.819092131 +0000 UTC m=+945.826408492 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/b7a108cc-ba0b-489e-ad11-7ee328edd6da-registry-tls") pod "feast-credit-scoring-747c6fb677-4h24k" (UID: "b7a108cc-ba0b-489e-ad11-7ee328edd6da") : secret "feast-credit-scoring-registry-tls" not found Sep 10 12:33:45.319278 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:45.319255 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b7a108cc-ba0b-489e-ad11-7ee328edd6da-feast-data\") pod \"feast-credit-scoring-747c6fb677-4h24k\" (UID: \"b7a108cc-ba0b-489e-ad11-7ee328edd6da\") " pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" Sep 10 12:33:45.321198 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:45.321182 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b7a108cc-ba0b-489e-ad11-7ee328edd6da-online-tls\") pod \"feast-credit-scoring-747c6fb677-4h24k\" (UID: \"b7a108cc-ba0b-489e-ad11-7ee328edd6da\") " pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" Sep 10 12:33:45.327981 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:45.327959 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6fzt\" (UniqueName: \"kubernetes.io/projected/b7a108cc-ba0b-489e-ad11-7ee328edd6da-kube-api-access-t6fzt\") pod \"feast-credit-scoring-747c6fb677-4h24k\" (UID: \"b7a108cc-ba0b-489e-ad11-7ee328edd6da\") " pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" Sep 10 12:33:45.822439 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:45.822409 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b7a108cc-ba0b-489e-ad11-7ee328edd6da-registry-tls\") pod \"feast-credit-scoring-747c6fb677-4h24k\" (UID: \"b7a108cc-ba0b-489e-ad11-7ee328edd6da\") " pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" Sep 10 12:33:45.824691 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:45.824662 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b7a108cc-ba0b-489e-ad11-7ee328edd6da-registry-tls\") pod \"feast-credit-scoring-747c6fb677-4h24k\" (UID: \"b7a108cc-ba0b-489e-ad11-7ee328edd6da\") " pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" Sep 10 12:33:45.999980 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:45.999950 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" Sep 10 12:33:46.276608 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:46.276586 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k"] Sep 10 12:33:46.278042 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:33:46.278015 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7a108cc_ba0b_489e_ad11_7ee328edd6da.slice/crio-b2438f23efe35d81f3b6bd247a3f1a51c88450fd332cfc8ae00e662eb65c1b81 WatchSource:0}: Error finding container b2438f23efe35d81f3b6bd247a3f1a51c88450fd332cfc8ae00e662eb65c1b81: Status 404 returned error can't find the container with id b2438f23efe35d81f3b6bd247a3f1a51c88450fd332cfc8ae00e662eb65c1b81 Sep 10 12:33:46.768924 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:46.768886 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" event={"ID":"b7a108cc-ba0b-489e-ad11-7ee328edd6da","Type":"ContainerStarted","Data":"841b63de8fd8ae339d52453daee3be390f5666d9aecfc495b2a6573c5378e4ad"} Sep 10 12:33:46.769094 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:46.768941 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" event={"ID":"b7a108cc-ba0b-489e-ad11-7ee328edd6da","Type":"ContainerStarted","Data":"b2438f23efe35d81f3b6bd247a3f1a51c88450fd332cfc8ae00e662eb65c1b81"} Sep 10 12:33:47.774287 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:47.774257 2580 generic.go:356] "Generic (PLEG): container finished" podID="b7a108cc-ba0b-489e-ad11-7ee328edd6da" containerID="841b63de8fd8ae339d52453daee3be390f5666d9aecfc495b2a6573c5378e4ad" exitCode=0 Sep 10 12:33:47.774752 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:47.774328 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" event={"ID":"b7a108cc-ba0b-489e-ad11-7ee328edd6da","Type":"ContainerDied","Data":"841b63de8fd8ae339d52453daee3be390f5666d9aecfc495b2a6573c5378e4ad"} Sep 10 12:33:48.780771 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:48.780730 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" event={"ID":"b7a108cc-ba0b-489e-ad11-7ee328edd6da","Type":"ContainerStarted","Data":"5d83bdefb12e381c235c4ee3b28054acdb114bfcc802dc41e2727af61bc05af8"} Sep 10 12:33:54.807710 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:54.807677 2580 generic.go:356] "Generic (PLEG): container finished" podID="b7a108cc-ba0b-489e-ad11-7ee328edd6da" containerID="5d83bdefb12e381c235c4ee3b28054acdb114bfcc802dc41e2727af61bc05af8" exitCode=0 Sep 10 12:33:54.808158 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:54.807747 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" event={"ID":"b7a108cc-ba0b-489e-ad11-7ee328edd6da","Type":"ContainerDied","Data":"5d83bdefb12e381c235c4ee3b28054acdb114bfcc802dc41e2727af61bc05af8"} Sep 10 12:33:55.815402 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:55.815367 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" event={"ID":"b7a108cc-ba0b-489e-ad11-7ee328edd6da","Type":"ContainerStarted","Data":"07be6b37cdd0eed94e0db0ceef8a2de8e733f852845379ff0dafe6636d36d0ae"} Sep 10 12:33:55.815953 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:55.815424 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" event={"ID":"b7a108cc-ba0b-489e-ad11-7ee328edd6da","Type":"ContainerStarted","Data":"cbc449d4be040a01d1a4a61487dc87f61e4e3f0911d7813251bad5fd59ca410a"} Sep 10 12:33:55.838261 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:55.838199 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" podStartSLOduration=10.838182503 podStartE2EDuration="10.838182503s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:33:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:33:55.836274578 +0000 UTC m=+955.843590980" watchObservedRunningTime="2026-09-10 12:33:55.838182503 +0000 UTC m=+955.845498886" Sep 10 12:33:56.000183 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:56.000147 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" Sep 10 12:33:56.000342 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:56.000220 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" Sep 10 12:33:58.000395 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:58.000364 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" Sep 10 12:33:58.000930 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:58.000429 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" Sep 10 12:33:58.002024 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:58.001992 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" podUID="b7a108cc-ba0b-489e-ad11-7ee328edd6da" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.27:6571: connect: connection refused" Sep 10 12:33:58.002024 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:33:58.002016 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" podUID="b7a108cc-ba0b-489e-ad11-7ee328edd6da" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused" Sep 10 12:34:01.001174 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:01.001122 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" podUID="b7a108cc-ba0b-489e-ad11-7ee328edd6da" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.27:6571: connect: connection refused" Sep 10 12:34:01.034904 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:01.034842 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" Sep 10 12:34:01.039387 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:01.039365 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" Sep 10 12:34:04.001063 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:04.001014 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" podUID="b7a108cc-ba0b-489e-ad11-7ee328edd6da" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.27:6571: connect: connection refused" Sep 10 12:34:07.000823 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:07.000788 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" Sep 10 12:34:07.001567 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:07.001547 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" Sep 10 12:34:10.975558 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:10.975526 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk"] Sep 10 12:34:10.979119 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:10.979102 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" Sep 10 12:34:10.981979 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:10.981949 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\"" Sep 10 12:34:10.981979 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:10.981964 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-registry-tls\"" Sep 10 12:34:10.981979 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:10.981975 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-online-tls\"" Sep 10 12:34:10.982185 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:10.982009 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-dockercfg-wnltm\"" Sep 10 12:34:10.989347 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:10.989327 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk"] Sep 10 12:34:11.153044 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:11.153005 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/31a3ac14-beac-4346-a27c-f3fbba35c9fa-online-tls\") pod \"feast-driver-ranking-65748b6878-w4vrk\" (UID: \"31a3ac14-beac-4346-a27c-f3fbba35c9fa\") " pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" Sep 10 12:34:11.153044 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:11.153040 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/31a3ac14-beac-4346-a27c-f3fbba35c9fa-feast-data\") pod \"feast-driver-ranking-65748b6878-w4vrk\" (UID: \"31a3ac14-beac-4346-a27c-f3fbba35c9fa\") " pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" Sep 10 12:34:11.153257 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:11.153064 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/31a3ac14-beac-4346-a27c-f3fbba35c9fa-registry-tls\") pod \"feast-driver-ranking-65748b6878-w4vrk\" (UID: \"31a3ac14-beac-4346-a27c-f3fbba35c9fa\") " pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" Sep 10 12:34:11.153257 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:11.153100 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nddqz\" (UniqueName: \"kubernetes.io/projected/31a3ac14-beac-4346-a27c-f3fbba35c9fa-kube-api-access-nddqz\") pod \"feast-driver-ranking-65748b6878-w4vrk\" (UID: \"31a3ac14-beac-4346-a27c-f3fbba35c9fa\") " pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" Sep 10 12:34:11.253931 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:11.253827 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/31a3ac14-beac-4346-a27c-f3fbba35c9fa-online-tls\") pod \"feast-driver-ranking-65748b6878-w4vrk\" (UID: \"31a3ac14-beac-4346-a27c-f3fbba35c9fa\") " pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" Sep 10 12:34:11.253931 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:11.253876 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/31a3ac14-beac-4346-a27c-f3fbba35c9fa-feast-data\") pod \"feast-driver-ranking-65748b6878-w4vrk\" (UID: \"31a3ac14-beac-4346-a27c-f3fbba35c9fa\") " pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" Sep 10 12:34:11.253931 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:11.253910 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/31a3ac14-beac-4346-a27c-f3fbba35c9fa-registry-tls\") pod \"feast-driver-ranking-65748b6878-w4vrk\" (UID: \"31a3ac14-beac-4346-a27c-f3fbba35c9fa\") " pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" Sep 10 12:34:11.254213 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:11.253944 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nddqz\" (UniqueName: \"kubernetes.io/projected/31a3ac14-beac-4346-a27c-f3fbba35c9fa-kube-api-access-nddqz\") pod \"feast-driver-ranking-65748b6878-w4vrk\" (UID: \"31a3ac14-beac-4346-a27c-f3fbba35c9fa\") " pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" Sep 10 12:34:11.254213 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:34:11.253987 2580 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-driver-ranking-online-tls: secret "feast-driver-ranking-online-tls" not found Sep 10 12:34:11.254213 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:34:11.254055 2580 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-driver-ranking-registry-tls: secret "feast-driver-ranking-registry-tls" not found Sep 10 12:34:11.254213 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:34:11.254135 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31a3ac14-beac-4346-a27c-f3fbba35c9fa-online-tls podName:31a3ac14-beac-4346-a27c-f3fbba35c9fa nodeName:}" failed. No retries permitted until 2026-09-10 12:34:11.754111728 +0000 UTC m=+971.761428111 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/31a3ac14-beac-4346-a27c-f3fbba35c9fa-online-tls") pod "feast-driver-ranking-65748b6878-w4vrk" (UID: "31a3ac14-beac-4346-a27c-f3fbba35c9fa") : secret "feast-driver-ranking-online-tls" not found Sep 10 12:34:11.254213 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:34:11.254174 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31a3ac14-beac-4346-a27c-f3fbba35c9fa-registry-tls podName:31a3ac14-beac-4346-a27c-f3fbba35c9fa nodeName:}" failed. No retries permitted until 2026-09-10 12:34:11.754148542 +0000 UTC m=+971.761464902 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/31a3ac14-beac-4346-a27c-f3fbba35c9fa-registry-tls") pod "feast-driver-ranking-65748b6878-w4vrk" (UID: "31a3ac14-beac-4346-a27c-f3fbba35c9fa") : secret "feast-driver-ranking-registry-tls" not found Sep 10 12:34:11.254412 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:11.254355 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/31a3ac14-beac-4346-a27c-f3fbba35c9fa-feast-data\") pod \"feast-driver-ranking-65748b6878-w4vrk\" (UID: \"31a3ac14-beac-4346-a27c-f3fbba35c9fa\") " pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" Sep 10 12:34:11.263322 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:11.263303 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nddqz\" (UniqueName: \"kubernetes.io/projected/31a3ac14-beac-4346-a27c-f3fbba35c9fa-kube-api-access-nddqz\") pod \"feast-driver-ranking-65748b6878-w4vrk\" (UID: \"31a3ac14-beac-4346-a27c-f3fbba35c9fa\") " pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" Sep 10 12:34:11.758727 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:11.758696 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/31a3ac14-beac-4346-a27c-f3fbba35c9fa-registry-tls\") pod \"feast-driver-ranking-65748b6878-w4vrk\" (UID: \"31a3ac14-beac-4346-a27c-f3fbba35c9fa\") " pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" Sep 10 12:34:11.758937 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:11.758788 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/31a3ac14-beac-4346-a27c-f3fbba35c9fa-online-tls\") pod \"feast-driver-ranking-65748b6878-w4vrk\" (UID: \"31a3ac14-beac-4346-a27c-f3fbba35c9fa\") " pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" Sep 10 12:34:11.761125 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:11.761096 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/31a3ac14-beac-4346-a27c-f3fbba35c9fa-registry-tls\") pod \"feast-driver-ranking-65748b6878-w4vrk\" (UID: \"31a3ac14-beac-4346-a27c-f3fbba35c9fa\") " pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" Sep 10 12:34:11.761232 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:11.761199 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/31a3ac14-beac-4346-a27c-f3fbba35c9fa-online-tls\") pod \"feast-driver-ranking-65748b6878-w4vrk\" (UID: \"31a3ac14-beac-4346-a27c-f3fbba35c9fa\") " pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" Sep 10 12:34:11.888632 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:11.888604 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" Sep 10 12:34:12.165172 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:12.165144 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk"] Sep 10 12:34:12.166007 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:34:12.165982 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31a3ac14_beac_4346_a27c_f3fbba35c9fa.slice/crio-6b36a87da3229c6f77bab63767b78acb3c174eebaa28fcbec8bf57088fb653ae WatchSource:0}: Error finding container 6b36a87da3229c6f77bab63767b78acb3c174eebaa28fcbec8bf57088fb653ae: Status 404 returned error can't find the container with id 6b36a87da3229c6f77bab63767b78acb3c174eebaa28fcbec8bf57088fb653ae Sep 10 12:34:12.891988 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:12.891948 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" event={"ID":"31a3ac14-beac-4346-a27c-f3fbba35c9fa","Type":"ContainerStarted","Data":"d0d9a178a3025db0cc5791125b01ababff31e6332b4e550cd6731a1e82d30ece"} Sep 10 12:34:12.892192 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:12.892011 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" event={"ID":"31a3ac14-beac-4346-a27c-f3fbba35c9fa","Type":"ContainerStarted","Data":"6b36a87da3229c6f77bab63767b78acb3c174eebaa28fcbec8bf57088fb653ae"} Sep 10 12:34:16.910895 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:16.910850 2580 generic.go:356] "Generic (PLEG): container finished" podID="31a3ac14-beac-4346-a27c-f3fbba35c9fa" containerID="d0d9a178a3025db0cc5791125b01ababff31e6332b4e550cd6731a1e82d30ece" exitCode=0 Sep 10 12:34:16.910895 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:16.910888 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" event={"ID":"31a3ac14-beac-4346-a27c-f3fbba35c9fa","Type":"ContainerDied","Data":"d0d9a178a3025db0cc5791125b01ababff31e6332b4e550cd6731a1e82d30ece"} Sep 10 12:34:17.916968 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:17.916926 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" event={"ID":"31a3ac14-beac-4346-a27c-f3fbba35c9fa","Type":"ContainerStarted","Data":"8f753dfe4a3e733ae0e21d036f412b321aa45334adf5560a80528b4159e299c1"} Sep 10 12:34:23.945055 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:23.945022 2580 generic.go:356] "Generic (PLEG): container finished" podID="31a3ac14-beac-4346-a27c-f3fbba35c9fa" containerID="8f753dfe4a3e733ae0e21d036f412b321aa45334adf5560a80528b4159e299c1" exitCode=0 Sep 10 12:34:23.945447 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:23.945091 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" event={"ID":"31a3ac14-beac-4346-a27c-f3fbba35c9fa","Type":"ContainerDied","Data":"8f753dfe4a3e733ae0e21d036f412b321aa45334adf5560a80528b4159e299c1"} Sep 10 12:34:24.956630 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:24.956599 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" event={"ID":"31a3ac14-beac-4346-a27c-f3fbba35c9fa","Type":"ContainerStarted","Data":"4ca4d8bb0987b725479feaaf3bb65d213e2a4575ffc6d51dd6120b75e075b900"} Sep 10 12:34:24.957128 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:24.956660 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" event={"ID":"31a3ac14-beac-4346-a27c-f3fbba35c9fa","Type":"ContainerStarted","Data":"fe9219723b2a38bdc0ea3c1120aac1e5344df3f87869be32305e27c2ce895d1d"} Sep 10 12:34:24.979011 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:24.978949 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" podStartSLOduration=14.978928353 podStartE2EDuration="14.978928353s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:34:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:34:24.976505883 +0000 UTC m=+984.983822288" watchObservedRunningTime="2026-09-10 12:34:24.978928353 +0000 UTC m=+984.986244736" Sep 10 12:34:26.888899 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:26.888846 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" Sep 10 12:34:26.889278 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:26.888928 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" Sep 10 12:34:26.890243 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:26.890213 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" podUID="31a3ac14-beac-4346-a27c-f3fbba35c9fa" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.28:6571: connect: connection refused" Sep 10 12:34:26.890346 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:26.890241 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" podUID="31a3ac14-beac-4346-a27c-f3fbba35c9fa" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused" Sep 10 12:34:29.889116 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:29.889067 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" podUID="31a3ac14-beac-4346-a27c-f3fbba35c9fa" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused" Sep 10 12:34:29.889557 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:29.889069 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" podUID="31a3ac14-beac-4346-a27c-f3fbba35c9fa" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.28:6571: connect: connection refused" Sep 10 12:34:31.888970 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:31.888928 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" Sep 10 12:34:31.889334 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:31.889013 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" Sep 10 12:34:32.889554 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:32.889500 2580 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" podUID="31a3ac14-beac-4346-a27c-f3fbba35c9fa" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.28:6571: connect: connection refused" Sep 10 12:34:32.895417 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:32.895388 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" Sep 10 12:34:32.898866 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:32.898832 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" Sep 10 12:34:35.889748 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:35.889716 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" Sep 10 12:34:35.890486 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:34:35.890470 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" Sep 10 12:35:53.161478 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:53.161405 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k"] Sep 10 12:35:53.161950 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:53.161632 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" podUID="b7a108cc-ba0b-489e-ad11-7ee328edd6da" containerName="registry" containerID="cri-o://cbc449d4be040a01d1a4a61487dc87f61e4e3f0911d7813251bad5fd59ca410a" gracePeriod=30 Sep 10 12:35:53.161950 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:53.161663 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" podUID="b7a108cc-ba0b-489e-ad11-7ee328edd6da" containerName="online" containerID="cri-o://07be6b37cdd0eed94e0db0ceef8a2de8e733f852845379ff0dafe6636d36d0ae" gracePeriod=30 Sep 10 12:35:53.166031 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:53.166005 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk"] Sep 10 12:35:53.166260 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:53.166225 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" podUID="31a3ac14-beac-4346-a27c-f3fbba35c9fa" containerName="registry" containerID="cri-o://fe9219723b2a38bdc0ea3c1120aac1e5344df3f87869be32305e27c2ce895d1d" gracePeriod=30 Sep 10 12:35:53.166360 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:53.166244 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" podUID="31a3ac14-beac-4346-a27c-f3fbba35c9fa" containerName="online" containerID="cri-o://4ca4d8bb0987b725479feaaf3bb65d213e2a4575ffc6d51dd6120b75e075b900" gracePeriod=30 Sep 10 12:35:54.359652 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:54.359622 2580 generic.go:356] "Generic (PLEG): container finished" podID="b7a108cc-ba0b-489e-ad11-7ee328edd6da" containerID="07be6b37cdd0eed94e0db0ceef8a2de8e733f852845379ff0dafe6636d36d0ae" exitCode=0 Sep 10 12:35:54.360074 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:54.359700 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" event={"ID":"b7a108cc-ba0b-489e-ad11-7ee328edd6da","Type":"ContainerDied","Data":"07be6b37cdd0eed94e0db0ceef8a2de8e733f852845379ff0dafe6636d36d0ae"} Sep 10 12:35:54.361801 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:54.361781 2580 generic.go:356] "Generic (PLEG): container finished" podID="31a3ac14-beac-4346-a27c-f3fbba35c9fa" containerID="4ca4d8bb0987b725479feaaf3bb65d213e2a4575ffc6d51dd6120b75e075b900" exitCode=0 Sep 10 12:35:54.361934 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:54.361812 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" event={"ID":"31a3ac14-beac-4346-a27c-f3fbba35c9fa","Type":"ContainerDied","Data":"4ca4d8bb0987b725479feaaf3bb65d213e2a4575ffc6d51dd6120b75e075b900"} Sep 10 12:35:55.551866 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:55.551807 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-5ndhg"] Sep 10 12:35:55.552338 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:55.552309 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/postgres-864846d4f5-5ndhg" podUID="5bd89a92-2371-4743-94ef-2907d0cc3d76" containerName="postgres" containerID="cri-o://2b4e8ceba3d48012807be08a01188f8e3ffa31cddfbfee6a8426a74898d74b38" gracePeriod=30 Sep 10 12:35:55.556815 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:55.556787 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-jpxt9"] Sep 10 12:35:55.557004 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:55.556976 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/redis-69bbd798b6-jpxt9" podUID="07d57c47-d5b0-43b5-b54e-5d0b7325ae7e" containerName="redis" containerID="cri-o://2c74d70a24bd0ae85c6dc7cd9517c30d6c0a841bd3f1d8fbd6dda01238a6698a" gracePeriod=30 Sep 10 12:35:55.812571 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:55.812549 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-jpxt9" Sep 10 12:35:55.815151 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:55.815133 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-5ndhg" Sep 10 12:35:55.884453 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:55.884427 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8bhgj\" (UniqueName: \"kubernetes.io/projected/07d57c47-d5b0-43b5-b54e-5d0b7325ae7e-kube-api-access-8bhgj\") pod \"07d57c47-d5b0-43b5-b54e-5d0b7325ae7e\" (UID: \"07d57c47-d5b0-43b5-b54e-5d0b7325ae7e\") " Sep 10 12:35:55.884598 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:55.884478 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6v6ln\" (UniqueName: \"kubernetes.io/projected/5bd89a92-2371-4743-94ef-2907d0cc3d76-kube-api-access-6v6ln\") pod \"5bd89a92-2371-4743-94ef-2907d0cc3d76\" (UID: \"5bd89a92-2371-4743-94ef-2907d0cc3d76\") " Sep 10 12:35:55.884598 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:55.884553 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/5bd89a92-2371-4743-94ef-2907d0cc3d76-postgresdata\") pod \"5bd89a92-2371-4743-94ef-2907d0cc3d76\" (UID: \"5bd89a92-2371-4743-94ef-2907d0cc3d76\") " Sep 10 12:35:55.886481 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:55.886451 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bd89a92-2371-4743-94ef-2907d0cc3d76-kube-api-access-6v6ln" (OuterVolumeSpecName: "kube-api-access-6v6ln") pod "5bd89a92-2371-4743-94ef-2907d0cc3d76" (UID: "5bd89a92-2371-4743-94ef-2907d0cc3d76"). InnerVolumeSpecName "kube-api-access-6v6ln". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:35:55.886614 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:55.886485 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07d57c47-d5b0-43b5-b54e-5d0b7325ae7e-kube-api-access-8bhgj" (OuterVolumeSpecName: "kube-api-access-8bhgj") pod "07d57c47-d5b0-43b5-b54e-5d0b7325ae7e" (UID: "07d57c47-d5b0-43b5-b54e-5d0b7325ae7e"). InnerVolumeSpecName "kube-api-access-8bhgj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:35:55.913643 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:55.913617 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bd89a92-2371-4743-94ef-2907d0cc3d76-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "5bd89a92-2371-4743-94ef-2907d0cc3d76" (UID: "5bd89a92-2371-4743-94ef-2907d0cc3d76"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 12:35:55.985443 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:55.985411 2580 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/5bd89a92-2371-4743-94ef-2907d0cc3d76-postgresdata\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:35:55.985443 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:55.985438 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8bhgj\" (UniqueName: \"kubernetes.io/projected/07d57c47-d5b0-43b5-b54e-5d0b7325ae7e-kube-api-access-8bhgj\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:35:55.985443 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:55.985448 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6v6ln\" (UniqueName: \"kubernetes.io/projected/5bd89a92-2371-4743-94ef-2907d0cc3d76-kube-api-access-6v6ln\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:35:56.370395 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:56.370360 2580 generic.go:356] "Generic (PLEG): container finished" podID="5bd89a92-2371-4743-94ef-2907d0cc3d76" containerID="2b4e8ceba3d48012807be08a01188f8e3ffa31cddfbfee6a8426a74898d74b38" exitCode=0 Sep 10 12:35:56.370556 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:56.370404 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-5ndhg" Sep 10 12:35:56.370556 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:56.370434 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-5ndhg" event={"ID":"5bd89a92-2371-4743-94ef-2907d0cc3d76","Type":"ContainerDied","Data":"2b4e8ceba3d48012807be08a01188f8e3ffa31cddfbfee6a8426a74898d74b38"} Sep 10 12:35:56.370556 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:56.370474 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-5ndhg" event={"ID":"5bd89a92-2371-4743-94ef-2907d0cc3d76","Type":"ContainerDied","Data":"e624a403f3eb74c02d8622f278c3fc81f9885c29ef845593969bb329f95fb1d0"} Sep 10 12:35:56.370556 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:56.370490 2580 scope.go:117] "RemoveContainer" containerID="2b4e8ceba3d48012807be08a01188f8e3ffa31cddfbfee6a8426a74898d74b38" Sep 10 12:35:56.371812 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:56.371787 2580 generic.go:356] "Generic (PLEG): container finished" podID="07d57c47-d5b0-43b5-b54e-5d0b7325ae7e" containerID="2c74d70a24bd0ae85c6dc7cd9517c30d6c0a841bd3f1d8fbd6dda01238a6698a" exitCode=0 Sep 10 12:35:56.371958 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:56.371830 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-jpxt9" Sep 10 12:35:56.371958 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:56.371840 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-jpxt9" event={"ID":"07d57c47-d5b0-43b5-b54e-5d0b7325ae7e","Type":"ContainerDied","Data":"2c74d70a24bd0ae85c6dc7cd9517c30d6c0a841bd3f1d8fbd6dda01238a6698a"} Sep 10 12:35:56.371958 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:56.371881 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-jpxt9" event={"ID":"07d57c47-d5b0-43b5-b54e-5d0b7325ae7e","Type":"ContainerDied","Data":"3eaf094ad2f41c538aa322088b6d673a1b9e583d6f8e23b96b51ef831c27d207"} Sep 10 12:35:56.380978 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:56.380841 2580 scope.go:117] "RemoveContainer" containerID="2b4e8ceba3d48012807be08a01188f8e3ffa31cddfbfee6a8426a74898d74b38" Sep 10 12:35:56.381159 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:35:56.381134 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b4e8ceba3d48012807be08a01188f8e3ffa31cddfbfee6a8426a74898d74b38\": container with ID starting with 2b4e8ceba3d48012807be08a01188f8e3ffa31cddfbfee6a8426a74898d74b38 not found: ID does not exist" containerID="2b4e8ceba3d48012807be08a01188f8e3ffa31cddfbfee6a8426a74898d74b38" Sep 10 12:35:56.381221 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:56.381171 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b4e8ceba3d48012807be08a01188f8e3ffa31cddfbfee6a8426a74898d74b38"} err="failed to get container status \"2b4e8ceba3d48012807be08a01188f8e3ffa31cddfbfee6a8426a74898d74b38\": rpc error: code = NotFound desc = could not find container \"2b4e8ceba3d48012807be08a01188f8e3ffa31cddfbfee6a8426a74898d74b38\": container with ID starting with 2b4e8ceba3d48012807be08a01188f8e3ffa31cddfbfee6a8426a74898d74b38 not found: ID does not exist" Sep 10 12:35:56.381221 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:56.381194 2580 scope.go:117] "RemoveContainer" containerID="2c74d70a24bd0ae85c6dc7cd9517c30d6c0a841bd3f1d8fbd6dda01238a6698a" Sep 10 12:35:56.388394 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:56.388361 2580 scope.go:117] "RemoveContainer" containerID="2c74d70a24bd0ae85c6dc7cd9517c30d6c0a841bd3f1d8fbd6dda01238a6698a" Sep 10 12:35:56.388679 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:35:56.388652 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c74d70a24bd0ae85c6dc7cd9517c30d6c0a841bd3f1d8fbd6dda01238a6698a\": container with ID starting with 2c74d70a24bd0ae85c6dc7cd9517c30d6c0a841bd3f1d8fbd6dda01238a6698a not found: ID does not exist" containerID="2c74d70a24bd0ae85c6dc7cd9517c30d6c0a841bd3f1d8fbd6dda01238a6698a" Sep 10 12:35:56.388679 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:56.388684 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c74d70a24bd0ae85c6dc7cd9517c30d6c0a841bd3f1d8fbd6dda01238a6698a"} err="failed to get container status \"2c74d70a24bd0ae85c6dc7cd9517c30d6c0a841bd3f1d8fbd6dda01238a6698a\": rpc error: code = NotFound desc = could not find container \"2c74d70a24bd0ae85c6dc7cd9517c30d6c0a841bd3f1d8fbd6dda01238a6698a\": container with ID starting with 2c74d70a24bd0ae85c6dc7cd9517c30d6c0a841bd3f1d8fbd6dda01238a6698a not found: ID does not exist" Sep 10 12:35:56.390633 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:56.390612 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-5ndhg"] Sep 10 12:35:56.392032 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:56.392016 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-5ndhg"] Sep 10 12:35:56.400655 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:56.400636 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-jpxt9"] Sep 10 12:35:56.402654 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:56.402633 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-jpxt9"] Sep 10 12:35:56.557160 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:56.557132 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07d57c47-d5b0-43b5-b54e-5d0b7325ae7e" path="/var/lib/kubelet/pods/07d57c47-d5b0-43b5-b54e-5d0b7325ae7e/volumes" Sep 10 12:35:56.557506 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:35:56.557481 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bd89a92-2371-4743-94ef-2907d0cc3d76" path="/var/lib/kubelet/pods/5bd89a92-2371-4743-94ef-2907d0cc3d76/volumes" Sep 10 12:36:01.035742 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:01.035698 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" podUID="b7a108cc-ba0b-489e-ad11-7ee328edd6da" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused" Sep 10 12:36:02.896744 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:02.896699 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" podUID="31a3ac14-beac-4346-a27c-f3fbba35c9fa" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused" Sep 10 12:36:11.036411 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:11.036365 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" podUID="b7a108cc-ba0b-489e-ad11-7ee328edd6da" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused" Sep 10 12:36:12.896081 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:12.896037 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" podUID="31a3ac14-beac-4346-a27c-f3fbba35c9fa" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused" Sep 10 12:36:21.035671 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:21.035619 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" podUID="b7a108cc-ba0b-489e-ad11-7ee328edd6da" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused" Sep 10 12:36:21.036080 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:21.035785 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" Sep 10 12:36:22.896318 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:22.896276 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" podUID="31a3ac14-beac-4346-a27c-f3fbba35c9fa" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused" Sep 10 12:36:22.896769 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:22.896382 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" Sep 10 12:36:23.486193 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:23.486109 2580 generic.go:356] "Generic (PLEG): container finished" podID="31a3ac14-beac-4346-a27c-f3fbba35c9fa" containerID="fe9219723b2a38bdc0ea3c1120aac1e5344df3f87869be32305e27c2ce895d1d" exitCode=137 Sep 10 12:36:23.486193 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:23.486183 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" event={"ID":"31a3ac14-beac-4346-a27c-f3fbba35c9fa","Type":"ContainerDied","Data":"fe9219723b2a38bdc0ea3c1120aac1e5344df3f87869be32305e27c2ce895d1d"} Sep 10 12:36:23.488433 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:23.488401 2580 generic.go:356] "Generic (PLEG): container finished" podID="b7a108cc-ba0b-489e-ad11-7ee328edd6da" containerID="cbc449d4be040a01d1a4a61487dc87f61e4e3f0911d7813251bad5fd59ca410a" exitCode=137 Sep 10 12:36:23.488546 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:23.488440 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" event={"ID":"b7a108cc-ba0b-489e-ad11-7ee328edd6da","Type":"ContainerDied","Data":"cbc449d4be040a01d1a4a61487dc87f61e4e3f0911d7813251bad5fd59ca410a"} Sep 10 12:36:23.825895 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:23.825853 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" Sep 10 12:36:23.828154 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:23.828135 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" Sep 10 12:36:23.938780 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:23.938752 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/31a3ac14-beac-4346-a27c-f3fbba35c9fa-online-tls\") pod \"31a3ac14-beac-4346-a27c-f3fbba35c9fa\" (UID: \"31a3ac14-beac-4346-a27c-f3fbba35c9fa\") " Sep 10 12:36:23.939363 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:23.938800 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b7a108cc-ba0b-489e-ad11-7ee328edd6da-registry-tls\") pod \"b7a108cc-ba0b-489e-ad11-7ee328edd6da\" (UID: \"b7a108cc-ba0b-489e-ad11-7ee328edd6da\") " Sep 10 12:36:23.939363 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:23.938838 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6fzt\" (UniqueName: \"kubernetes.io/projected/b7a108cc-ba0b-489e-ad11-7ee328edd6da-kube-api-access-t6fzt\") pod \"b7a108cc-ba0b-489e-ad11-7ee328edd6da\" (UID: \"b7a108cc-ba0b-489e-ad11-7ee328edd6da\") " Sep 10 12:36:23.939363 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:23.938908 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nddqz\" (UniqueName: \"kubernetes.io/projected/31a3ac14-beac-4346-a27c-f3fbba35c9fa-kube-api-access-nddqz\") pod \"31a3ac14-beac-4346-a27c-f3fbba35c9fa\" (UID: \"31a3ac14-beac-4346-a27c-f3fbba35c9fa\") " Sep 10 12:36:23.939363 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:23.938932 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b7a108cc-ba0b-489e-ad11-7ee328edd6da-feast-data\") pod \"b7a108cc-ba0b-489e-ad11-7ee328edd6da\" (UID: \"b7a108cc-ba0b-489e-ad11-7ee328edd6da\") " Sep 10 12:36:23.939363 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:23.939142 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/31a3ac14-beac-4346-a27c-f3fbba35c9fa-registry-tls\") pod \"31a3ac14-beac-4346-a27c-f3fbba35c9fa\" (UID: \"31a3ac14-beac-4346-a27c-f3fbba35c9fa\") " Sep 10 12:36:23.939363 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:23.939191 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b7a108cc-ba0b-489e-ad11-7ee328edd6da-online-tls\") pod \"b7a108cc-ba0b-489e-ad11-7ee328edd6da\" (UID: \"b7a108cc-ba0b-489e-ad11-7ee328edd6da\") " Sep 10 12:36:23.939363 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:23.939240 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/31a3ac14-beac-4346-a27c-f3fbba35c9fa-feast-data\") pod \"31a3ac14-beac-4346-a27c-f3fbba35c9fa\" (UID: \"31a3ac14-beac-4346-a27c-f3fbba35c9fa\") " Sep 10 12:36:23.940028 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:23.939955 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31a3ac14-beac-4346-a27c-f3fbba35c9fa-feast-data" (OuterVolumeSpecName: "feast-data") pod "31a3ac14-beac-4346-a27c-f3fbba35c9fa" (UID: "31a3ac14-beac-4346-a27c-f3fbba35c9fa"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 12:36:23.941443 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:23.941398 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7a108cc-ba0b-489e-ad11-7ee328edd6da-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "b7a108cc-ba0b-489e-ad11-7ee328edd6da" (UID: "b7a108cc-ba0b-489e-ad11-7ee328edd6da"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:36:23.941664 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:23.941639 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31a3ac14-beac-4346-a27c-f3fbba35c9fa-kube-api-access-nddqz" (OuterVolumeSpecName: "kube-api-access-nddqz") pod "31a3ac14-beac-4346-a27c-f3fbba35c9fa" (UID: "31a3ac14-beac-4346-a27c-f3fbba35c9fa"). InnerVolumeSpecName "kube-api-access-nddqz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:36:23.941762 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:23.941709 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7a108cc-ba0b-489e-ad11-7ee328edd6da-kube-api-access-t6fzt" (OuterVolumeSpecName: "kube-api-access-t6fzt") pod "b7a108cc-ba0b-489e-ad11-7ee328edd6da" (UID: "b7a108cc-ba0b-489e-ad11-7ee328edd6da"). InnerVolumeSpecName "kube-api-access-t6fzt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:36:23.941762 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:23.941719 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31a3ac14-beac-4346-a27c-f3fbba35c9fa-online-tls" (OuterVolumeSpecName: "online-tls") pod "31a3ac14-beac-4346-a27c-f3fbba35c9fa" (UID: "31a3ac14-beac-4346-a27c-f3fbba35c9fa"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:36:23.941893 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:23.941790 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31a3ac14-beac-4346-a27c-f3fbba35c9fa-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "31a3ac14-beac-4346-a27c-f3fbba35c9fa" (UID: "31a3ac14-beac-4346-a27c-f3fbba35c9fa"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:36:23.941893 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:23.941835 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7a108cc-ba0b-489e-ad11-7ee328edd6da-online-tls" (OuterVolumeSpecName: "online-tls") pod "b7a108cc-ba0b-489e-ad11-7ee328edd6da" (UID: "b7a108cc-ba0b-489e-ad11-7ee328edd6da"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:36:23.948093 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:23.948071 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7a108cc-ba0b-489e-ad11-7ee328edd6da-feast-data" (OuterVolumeSpecName: "feast-data") pod "b7a108cc-ba0b-489e-ad11-7ee328edd6da" (UID: "b7a108cc-ba0b-489e-ad11-7ee328edd6da"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 12:36:24.040279 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:24.040195 2580 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/31a3ac14-beac-4346-a27c-f3fbba35c9fa-feast-data\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:36:24.040279 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:24.040224 2580 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/31a3ac14-beac-4346-a27c-f3fbba35c9fa-online-tls\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:36:24.040279 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:24.040232 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b7a108cc-ba0b-489e-ad11-7ee328edd6da-registry-tls\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:36:24.040279 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:24.040240 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6fzt\" (UniqueName: \"kubernetes.io/projected/b7a108cc-ba0b-489e-ad11-7ee328edd6da-kube-api-access-t6fzt\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:36:24.040279 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:24.040250 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nddqz\" (UniqueName: \"kubernetes.io/projected/31a3ac14-beac-4346-a27c-f3fbba35c9fa-kube-api-access-nddqz\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:36:24.040279 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:24.040258 2580 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b7a108cc-ba0b-489e-ad11-7ee328edd6da-feast-data\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:36:24.040279 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:24.040266 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/31a3ac14-beac-4346-a27c-f3fbba35c9fa-registry-tls\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:36:24.040279 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:24.040273 2580 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b7a108cc-ba0b-489e-ad11-7ee328edd6da-online-tls\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:36:24.495146 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:24.495117 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" Sep 10 12:36:24.495338 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:24.495122 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk" event={"ID":"31a3ac14-beac-4346-a27c-f3fbba35c9fa","Type":"ContainerDied","Data":"6b36a87da3229c6f77bab63767b78acb3c174eebaa28fcbec8bf57088fb653ae"} Sep 10 12:36:24.495338 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:24.495263 2580 scope.go:117] "RemoveContainer" containerID="4ca4d8bb0987b725479feaaf3bb65d213e2a4575ffc6d51dd6120b75e075b900" Sep 10 12:36:24.497491 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:24.497468 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" event={"ID":"b7a108cc-ba0b-489e-ad11-7ee328edd6da","Type":"ContainerDied","Data":"b2438f23efe35d81f3b6bd247a3f1a51c88450fd332cfc8ae00e662eb65c1b81"} Sep 10 12:36:24.497583 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:24.497506 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k" Sep 10 12:36:24.504484 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:24.504468 2580 scope.go:117] "RemoveContainer" containerID="fe9219723b2a38bdc0ea3c1120aac1e5344df3f87869be32305e27c2ce895d1d" Sep 10 12:36:24.512468 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:24.512450 2580 scope.go:117] "RemoveContainer" containerID="8f753dfe4a3e733ae0e21d036f412b321aa45334adf5560a80528b4159e299c1" Sep 10 12:36:24.519528 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:24.519503 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk"] Sep 10 12:36:24.519937 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:24.519897 2580 scope.go:117] "RemoveContainer" containerID="d0d9a178a3025db0cc5791125b01ababff31e6332b4e550cd6731a1e82d30ece" Sep 10 12:36:24.523932 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:24.523907 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-65748b6878-w4vrk"] Sep 10 12:36:24.527597 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:24.527581 2580 scope.go:117] "RemoveContainer" containerID="07be6b37cdd0eed94e0db0ceef8a2de8e733f852845379ff0dafe6636d36d0ae" Sep 10 12:36:24.534999 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:24.534979 2580 scope.go:117] "RemoveContainer" containerID="cbc449d4be040a01d1a4a61487dc87f61e4e3f0911d7813251bad5fd59ca410a" Sep 10 12:36:24.536553 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:24.536534 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k"] Sep 10 12:36:24.541285 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:24.541262 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-747c6fb677-4h24k"] Sep 10 12:36:24.543932 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:24.543912 2580 scope.go:117] "RemoveContainer" containerID="5d83bdefb12e381c235c4ee3b28054acdb114bfcc802dc41e2727af61bc05af8" Sep 10 12:36:24.551094 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:24.551078 2580 scope.go:117] "RemoveContainer" containerID="841b63de8fd8ae339d52453daee3be390f5666d9aecfc495b2a6573c5378e4ad" Sep 10 12:36:24.556939 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:24.556910 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31a3ac14-beac-4346-a27c-f3fbba35c9fa" path="/var/lib/kubelet/pods/31a3ac14-beac-4346-a27c-f3fbba35c9fa/volumes" Sep 10 12:36:24.557656 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:36:24.557634 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7a108cc-ba0b-489e-ad11-7ee328edd6da" path="/var/lib/kubelet/pods/b7a108cc-ba0b-489e-ad11-7ee328edd6da/volumes" Sep 10 12:38:00.502611 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:38:00.502584 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tfcmz_e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58/ovn-acl-logging/0.log" Sep 10 12:38:00.504753 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:38:00.504731 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tfcmz_e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58/ovn-acl-logging/0.log" Sep 10 12:43:00.529992 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:43:00.529963 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tfcmz_e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58/ovn-acl-logging/0.log" Sep 10 12:43:00.533094 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:43:00.533075 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tfcmz_e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58/ovn-acl-logging/0.log" Sep 10 12:47:40.616975 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.616939 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-2tmkq/must-gather-nl5h6"] Sep 10 12:47:40.617432 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.617243 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07d57c47-d5b0-43b5-b54e-5d0b7325ae7e" containerName="redis" Sep 10 12:47:40.617432 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.617253 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="07d57c47-d5b0-43b5-b54e-5d0b7325ae7e" containerName="redis" Sep 10 12:47:40.617432 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.617261 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31a3ac14-beac-4346-a27c-f3fbba35c9fa" containerName="online" Sep 10 12:47:40.617432 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.617268 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="31a3ac14-beac-4346-a27c-f3fbba35c9fa" containerName="online" Sep 10 12:47:40.617432 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.617284 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31a3ac14-beac-4346-a27c-f3fbba35c9fa" containerName="feast-apply" Sep 10 12:47:40.617432 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.617289 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="31a3ac14-beac-4346-a27c-f3fbba35c9fa" containerName="feast-apply" Sep 10 12:47:40.617432 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.617299 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7a108cc-ba0b-489e-ad11-7ee328edd6da" containerName="feast-init" Sep 10 12:47:40.617432 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.617304 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7a108cc-ba0b-489e-ad11-7ee328edd6da" containerName="feast-init" Sep 10 12:47:40.617432 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.617313 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7a108cc-ba0b-489e-ad11-7ee328edd6da" containerName="online" Sep 10 12:47:40.617432 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.617318 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7a108cc-ba0b-489e-ad11-7ee328edd6da" containerName="online" Sep 10 12:47:40.617432 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.617325 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7a108cc-ba0b-489e-ad11-7ee328edd6da" containerName="registry" Sep 10 12:47:40.617432 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.617330 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7a108cc-ba0b-489e-ad11-7ee328edd6da" containerName="registry" Sep 10 12:47:40.617432 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.617338 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31a3ac14-beac-4346-a27c-f3fbba35c9fa" containerName="feast-init" Sep 10 12:47:40.617432 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.617343 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="31a3ac14-beac-4346-a27c-f3fbba35c9fa" containerName="feast-init" Sep 10 12:47:40.617432 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.617348 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7a108cc-ba0b-489e-ad11-7ee328edd6da" containerName="feast-apply" Sep 10 12:47:40.617432 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.617352 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7a108cc-ba0b-489e-ad11-7ee328edd6da" containerName="feast-apply" Sep 10 12:47:40.617432 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.617356 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31a3ac14-beac-4346-a27c-f3fbba35c9fa" containerName="registry" Sep 10 12:47:40.617432 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.617361 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="31a3ac14-beac-4346-a27c-f3fbba35c9fa" containerName="registry" Sep 10 12:47:40.617432 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.617368 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bd89a92-2371-4743-94ef-2907d0cc3d76" containerName="postgres" Sep 10 12:47:40.617432 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.617372 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bd89a92-2371-4743-94ef-2907d0cc3d76" containerName="postgres" Sep 10 12:47:40.617432 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.617431 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bd89a92-2371-4743-94ef-2907d0cc3d76" containerName="postgres" Sep 10 12:47:40.617432 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.617439 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7a108cc-ba0b-489e-ad11-7ee328edd6da" containerName="online" Sep 10 12:47:40.618299 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.617445 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="31a3ac14-beac-4346-a27c-f3fbba35c9fa" containerName="registry" Sep 10 12:47:40.618299 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.617453 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="07d57c47-d5b0-43b5-b54e-5d0b7325ae7e" containerName="redis" Sep 10 12:47:40.618299 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.617459 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7a108cc-ba0b-489e-ad11-7ee328edd6da" containerName="registry" Sep 10 12:47:40.618299 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.617464 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="31a3ac14-beac-4346-a27c-f3fbba35c9fa" containerName="online" Sep 10 12:47:40.620522 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.620507 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2tmkq/must-gather-nl5h6" Sep 10 12:47:40.623633 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.623615 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2tmkq\"/\"openshift-service-ca.crt\"" Sep 10 12:47:40.623766 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.623667 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-2tmkq\"/\"default-dockercfg-ckdqf\"" Sep 10 12:47:40.624642 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.624626 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2tmkq\"/\"kube-root-ca.crt\"" Sep 10 12:47:40.628792 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.628771 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2tmkq/must-gather-nl5h6"] Sep 10 12:47:40.699001 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.698972 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a1274476-5f28-4d24-b7ea-215698c88764-must-gather-output\") pod \"must-gather-nl5h6\" (UID: \"a1274476-5f28-4d24-b7ea-215698c88764\") " pod="openshift-must-gather-2tmkq/must-gather-nl5h6" Sep 10 12:47:40.699143 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.699018 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlc7v\" (UniqueName: \"kubernetes.io/projected/a1274476-5f28-4d24-b7ea-215698c88764-kube-api-access-jlc7v\") pod \"must-gather-nl5h6\" (UID: \"a1274476-5f28-4d24-b7ea-215698c88764\") " pod="openshift-must-gather-2tmkq/must-gather-nl5h6" Sep 10 12:47:40.800291 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.800261 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a1274476-5f28-4d24-b7ea-215698c88764-must-gather-output\") pod \"must-gather-nl5h6\" (UID: \"a1274476-5f28-4d24-b7ea-215698c88764\") " pod="openshift-must-gather-2tmkq/must-gather-nl5h6" Sep 10 12:47:40.800462 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.800366 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlc7v\" (UniqueName: \"kubernetes.io/projected/a1274476-5f28-4d24-b7ea-215698c88764-kube-api-access-jlc7v\") pod \"must-gather-nl5h6\" (UID: \"a1274476-5f28-4d24-b7ea-215698c88764\") " pod="openshift-must-gather-2tmkq/must-gather-nl5h6" Sep 10 12:47:40.800594 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.800577 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a1274476-5f28-4d24-b7ea-215698c88764-must-gather-output\") pod \"must-gather-nl5h6\" (UID: \"a1274476-5f28-4d24-b7ea-215698c88764\") " pod="openshift-must-gather-2tmkq/must-gather-nl5h6" Sep 10 12:47:40.808849 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.808830 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlc7v\" (UniqueName: \"kubernetes.io/projected/a1274476-5f28-4d24-b7ea-215698c88764-kube-api-access-jlc7v\") pod \"must-gather-nl5h6\" (UID: \"a1274476-5f28-4d24-b7ea-215698c88764\") " pod="openshift-must-gather-2tmkq/must-gather-nl5h6" Sep 10 12:47:40.940761 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:40.940735 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2tmkq/must-gather-nl5h6" Sep 10 12:47:41.208891 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:41.208788 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2tmkq/must-gather-nl5h6"] Sep 10 12:47:41.211663 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:47:41.211635 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda1274476_5f28_4d24_b7ea_215698c88764.slice/crio-76b2eabb801f35d492a45f446527c417e4fffbccf800bd9ee5e5f4b1a760213f WatchSource:0}: Error finding container 76b2eabb801f35d492a45f446527c417e4fffbccf800bd9ee5e5f4b1a760213f: Status 404 returned error can't find the container with id 76b2eabb801f35d492a45f446527c417e4fffbccf800bd9ee5e5f4b1a760213f Sep 10 12:47:41.213420 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:41.213398 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 12:47:42.130618 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:42.130542 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2tmkq/must-gather-nl5h6" event={"ID":"a1274476-5f28-4d24-b7ea-215698c88764","Type":"ContainerStarted","Data":"76b2eabb801f35d492a45f446527c417e4fffbccf800bd9ee5e5f4b1a760213f"} Sep 10 12:47:48.160929 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:48.160896 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2tmkq/must-gather-nl5h6" event={"ID":"a1274476-5f28-4d24-b7ea-215698c88764","Type":"ContainerStarted","Data":"720169021b236d6bac84a390dba1a2d84626344bf7191fab49d6b31def43bb64"} Sep 10 12:47:48.161308 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:48.160946 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2tmkq/must-gather-nl5h6" event={"ID":"a1274476-5f28-4d24-b7ea-215698c88764","Type":"ContainerStarted","Data":"fdd30197ee02947a7750a0b5d05e112458e66ea08d2cdf2003415414c2cf439e"} Sep 10 12:47:48.179108 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:48.179058 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-2tmkq/must-gather-nl5h6" podStartSLOduration=2.124231815 podStartE2EDuration="8.179039629s" totalImagesPullingTime="6.054807814s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:47:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:47:48.177678725 +0000 UTC m=+1788.184995110" watchObservedRunningTime="2026-09-10 12:47:48.179039629 +0000 UTC m=+1788.186356012" Sep 10 12:47:57.200007 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:57.199973 2580 generic.go:356] "Generic (PLEG): container finished" podID="a1274476-5f28-4d24-b7ea-215698c88764" containerID="fdd30197ee02947a7750a0b5d05e112458e66ea08d2cdf2003415414c2cf439e" exitCode=0 Sep 10 12:47:57.200400 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:57.200030 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2tmkq/must-gather-nl5h6" event={"ID":"a1274476-5f28-4d24-b7ea-215698c88764","Type":"ContainerDied","Data":"fdd30197ee02947a7750a0b5d05e112458e66ea08d2cdf2003415414c2cf439e"} Sep 10 12:47:57.200400 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:57.200357 2580 scope.go:117] "RemoveContainer" containerID="fdd30197ee02947a7750a0b5d05e112458e66ea08d2cdf2003415414c2cf439e" Sep 10 12:47:57.713043 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:47:57.712981 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-2tmkq_must-gather-nl5h6_a1274476-5f28-4d24-b7ea-215698c88764/gather/0.log" Sep 10 12:48:00.563836 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:00.563809 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tfcmz_e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58/ovn-acl-logging/0.log" Sep 10 12:48:00.565851 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:00.565832 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tfcmz_e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58/ovn-acl-logging/0.log" Sep 10 12:48:00.984542 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:00.984511 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-ljkj8_67095b3f-a8e2-494b-9f83-b4eb0b5634a2/global-pull-secret-syncer/0.log" Sep 10 12:48:01.082248 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:01.082218 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-zjw9d_3cf0d46b-95a9-4187-9395-ebd8a11b5771/konnectivity-agent/0.log" Sep 10 12:48:01.103053 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:01.103030 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-221.ec2.internal_dbeff53fae53b2340a3f2a4afbf5b2aa/haproxy/0.log" Sep 10 12:48:03.045405 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:03.045369 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-2tmkq/must-gather-nl5h6"] Sep 10 12:48:03.045929 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:03.045572 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-2tmkq/must-gather-nl5h6" podUID="a1274476-5f28-4d24-b7ea-215698c88764" containerName="copy" containerID="cri-o://720169021b236d6bac84a390dba1a2d84626344bf7191fab49d6b31def43bb64" gracePeriod=2 Sep 10 12:48:03.047768 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:48:03.047734 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-nl5h6\" is forbidden: User \"system:node:ip-10-0-130-221.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-2tmkq\": no relationship found between node 'ip-10-0-130-221.ec2.internal' and this object" podUID="a1274476-5f28-4d24-b7ea-215698c88764" pod="openshift-must-gather-2tmkq/must-gather-nl5h6" Sep 10 12:48:03.048075 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:03.048055 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-2tmkq/must-gather-nl5h6"] Sep 10 12:48:03.225952 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:03.225930 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-2tmkq_must-gather-nl5h6_a1274476-5f28-4d24-b7ea-215698c88764/copy/0.log" Sep 10 12:48:03.226355 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:03.226319 2580 generic.go:356] "Generic (PLEG): container finished" podID="a1274476-5f28-4d24-b7ea-215698c88764" containerID="720169021b236d6bac84a390dba1a2d84626344bf7191fab49d6b31def43bb64" exitCode=143 Sep 10 12:48:03.269040 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:03.269020 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-2tmkq_must-gather-nl5h6_a1274476-5f28-4d24-b7ea-215698c88764/copy/0.log" Sep 10 12:48:03.270409 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:03.270394 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2tmkq/must-gather-nl5h6" Sep 10 12:48:03.272996 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:48:03.272973 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-nl5h6\" is forbidden: User \"system:node:ip-10-0-130-221.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-2tmkq\": no relationship found between node 'ip-10-0-130-221.ec2.internal' and this object" podUID="a1274476-5f28-4d24-b7ea-215698c88764" pod="openshift-must-gather-2tmkq/must-gather-nl5h6" Sep 10 12:48:03.402960 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:03.402890 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a1274476-5f28-4d24-b7ea-215698c88764-must-gather-output\") pod \"a1274476-5f28-4d24-b7ea-215698c88764\" (UID: \"a1274476-5f28-4d24-b7ea-215698c88764\") " Sep 10 12:48:03.403092 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:03.402973 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jlc7v\" (UniqueName: \"kubernetes.io/projected/a1274476-5f28-4d24-b7ea-215698c88764-kube-api-access-jlc7v\") pod \"a1274476-5f28-4d24-b7ea-215698c88764\" (UID: \"a1274476-5f28-4d24-b7ea-215698c88764\") " Sep 10 12:48:03.403242 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:03.403212 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1274476-5f28-4d24-b7ea-215698c88764-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "a1274476-5f28-4d24-b7ea-215698c88764" (UID: "a1274476-5f28-4d24-b7ea-215698c88764"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 12:48:03.404955 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:03.404933 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1274476-5f28-4d24-b7ea-215698c88764-kube-api-access-jlc7v" (OuterVolumeSpecName: "kube-api-access-jlc7v") pod "a1274476-5f28-4d24-b7ea-215698c88764" (UID: "a1274476-5f28-4d24-b7ea-215698c88764"). InnerVolumeSpecName "kube-api-access-jlc7v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:48:03.503653 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:03.503617 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlc7v\" (UniqueName: \"kubernetes.io/projected/a1274476-5f28-4d24-b7ea-215698c88764-kube-api-access-jlc7v\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:48:03.503653 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:03.503644 2580 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a1274476-5f28-4d24-b7ea-215698c88764-must-gather-output\") on node \"ip-10-0-130-221.ec2.internal\" DevicePath \"\"" Sep 10 12:48:04.231354 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:04.231328 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-2tmkq_must-gather-nl5h6_a1274476-5f28-4d24-b7ea-215698c88764/copy/0.log" Sep 10 12:48:04.231769 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:04.231712 2580 scope.go:117] "RemoveContainer" containerID="720169021b236d6bac84a390dba1a2d84626344bf7191fab49d6b31def43bb64" Sep 10 12:48:04.231769 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:04.231731 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2tmkq/must-gather-nl5h6" Sep 10 12:48:04.234209 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:48:04.234179 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-nl5h6\" is forbidden: User \"system:node:ip-10-0-130-221.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-2tmkq\": no relationship found between node 'ip-10-0-130-221.ec2.internal' and this object" podUID="a1274476-5f28-4d24-b7ea-215698c88764" pod="openshift-must-gather-2tmkq/must-gather-nl5h6" Sep 10 12:48:04.239190 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:04.239175 2580 scope.go:117] "RemoveContainer" containerID="fdd30197ee02947a7750a0b5d05e112458e66ea08d2cdf2003415414c2cf439e" Sep 10 12:48:04.241051 ip-10-0-130-221 kubenswrapper[2580]: E0910 12:48:04.241028 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-nl5h6\" is forbidden: User \"system:node:ip-10-0-130-221.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-2tmkq\": no relationship found between node 'ip-10-0-130-221.ec2.internal' and this object" podUID="a1274476-5f28-4d24-b7ea-215698c88764" pod="openshift-must-gather-2tmkq/must-gather-nl5h6" Sep 10 12:48:04.557547 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:04.557472 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1274476-5f28-4d24-b7ea-215698c88764" path="/var/lib/kubelet/pods/a1274476-5f28-4d24-b7ea-215698c88764/volumes" Sep 10 12:48:04.960175 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:04.960131 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f1f6afd1-7472-4c06-8651-f15801fab26d/alertmanager/0.log" Sep 10 12:48:04.989219 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:04.989187 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f1f6afd1-7472-4c06-8651-f15801fab26d/config-reloader/0.log" Sep 10 12:48:05.008628 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:05.008604 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f1f6afd1-7472-4c06-8651-f15801fab26d/kube-rbac-proxy-web/0.log" Sep 10 12:48:05.029339 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:05.029318 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f1f6afd1-7472-4c06-8651-f15801fab26d/kube-rbac-proxy/0.log" Sep 10 12:48:05.055386 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:05.055361 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f1f6afd1-7472-4c06-8651-f15801fab26d/kube-rbac-proxy-metric/0.log" Sep 10 12:48:05.093810 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:05.093786 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f1f6afd1-7472-4c06-8651-f15801fab26d/prom-label-proxy/0.log" Sep 10 12:48:05.135466 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:05.135442 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f1f6afd1-7472-4c06-8651-f15801fab26d/init-config-reloader/0.log" Sep 10 12:48:05.208775 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:05.208749 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-krrfz_226b1299-cb0c-4568-8ae4-b3d955914d15/kube-state-metrics/0.log" Sep 10 12:48:05.228689 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:05.228623 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-krrfz_226b1299-cb0c-4568-8ae4-b3d955914d15/kube-rbac-proxy-main/0.log" Sep 10 12:48:05.257758 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:05.257735 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-krrfz_226b1299-cb0c-4568-8ae4-b3d955914d15/kube-rbac-proxy-self/0.log" Sep 10 12:48:05.289837 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:05.289815 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5bbdc6f8cb-sck44_a17142e5-cea9-4e49-a808-f1310b79ea4c/metrics-server/0.log" Sep 10 12:48:05.323689 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:05.323664 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-d4hlf_41360c8c-a2f7-4c31-84c5-c92265753526/monitoring-plugin/0.log" Sep 10 12:48:05.355417 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:05.355400 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b6452_807c357a-a718-4e5e-8af5-29f075c8c276/node-exporter/0.log" Sep 10 12:48:05.375964 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:05.375944 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b6452_807c357a-a718-4e5e-8af5-29f075c8c276/kube-rbac-proxy/0.log" Sep 10 12:48:05.401986 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:05.401968 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b6452_807c357a-a718-4e5e-8af5-29f075c8c276/init-textfile/0.log" Sep 10 12:48:05.578091 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:05.578067 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-wxwzn_464a50e2-1786-451d-b7c5-567d20b5972f/kube-rbac-proxy-main/0.log" Sep 10 12:48:05.606745 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:05.606716 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-wxwzn_464a50e2-1786-451d-b7c5-567d20b5972f/kube-rbac-proxy-self/0.log" Sep 10 12:48:05.630963 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:05.630932 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-wxwzn_464a50e2-1786-451d-b7c5-567d20b5972f/openshift-state-metrics/0.log" Sep 10 12:48:05.679660 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:05.679637 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_98426a97-564b-45a6-b15e-6a018de393d5/prometheus/0.log" Sep 10 12:48:05.705303 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:05.705281 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_98426a97-564b-45a6-b15e-6a018de393d5/config-reloader/0.log" Sep 10 12:48:05.727835 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:05.727809 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_98426a97-564b-45a6-b15e-6a018de393d5/thanos-sidecar/0.log" Sep 10 12:48:05.747649 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:05.747629 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_98426a97-564b-45a6-b15e-6a018de393d5/kube-rbac-proxy-web/0.log" Sep 10 12:48:05.770419 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:05.770367 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_98426a97-564b-45a6-b15e-6a018de393d5/kube-rbac-proxy/0.log" Sep 10 12:48:05.792773 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:05.792752 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_98426a97-564b-45a6-b15e-6a018de393d5/kube-rbac-proxy-thanos/0.log" Sep 10 12:48:05.818870 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:05.818840 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_98426a97-564b-45a6-b15e-6a018de393d5/init-config-reloader/0.log" Sep 10 12:48:05.851611 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:05.851592 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-npsn2_7f6e361b-caa6-49bb-876e-a21a57badd11/prometheus-operator/0.log" Sep 10 12:48:05.876994 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:05.876979 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-npsn2_7f6e361b-caa6-49bb-876e-a21a57badd11/kube-rbac-proxy/0.log" Sep 10 12:48:05.901632 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:05.901610 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-nt7ht_5994acc2-09f9-40fe-835f-07f49c9fec9d/prometheus-operator-admission-webhook/0.log" Sep 10 12:48:05.953702 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:05.953679 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-79cb7874c6-brgtb_ca0765fc-d4d3-43fd-acae-3f0e0b3f5971/telemeter-client/0.log" Sep 10 12:48:05.983112 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:05.983092 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-79cb7874c6-brgtb_ca0765fc-d4d3-43fd-acae-3f0e0b3f5971/reload/0.log" Sep 10 12:48:06.011828 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:06.011807 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-79cb7874c6-brgtb_ca0765fc-d4d3-43fd-acae-3f0e0b3f5971/kube-rbac-proxy/0.log" Sep 10 12:48:06.055095 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:06.055035 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-57c946f66b-qt62h_6675777c-d2f9-4754-8531-548d9088d313/thanos-query/0.log" Sep 10 12:48:06.093822 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:06.093800 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-57c946f66b-qt62h_6675777c-d2f9-4754-8531-548d9088d313/kube-rbac-proxy-web/0.log" Sep 10 12:48:06.120964 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:06.120936 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-57c946f66b-qt62h_6675777c-d2f9-4754-8531-548d9088d313/kube-rbac-proxy/0.log" Sep 10 12:48:06.152290 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:06.152268 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-57c946f66b-qt62h_6675777c-d2f9-4754-8531-548d9088d313/prom-label-proxy/0.log" Sep 10 12:48:06.182265 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:06.182237 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-57c946f66b-qt62h_6675777c-d2f9-4754-8531-548d9088d313/kube-rbac-proxy-rules/0.log" Sep 10 12:48:06.214516 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:06.214496 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-57c946f66b-qt62h_6675777c-d2f9-4754-8531-548d9088d313/kube-rbac-proxy-metrics/0.log" Sep 10 12:48:08.488995 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:08.488964 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dxwh5/perf-node-gather-daemonset-5f526"] Sep 10 12:48:08.489370 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:08.489281 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1274476-5f28-4d24-b7ea-215698c88764" containerName="copy" Sep 10 12:48:08.489370 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:08.489293 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1274476-5f28-4d24-b7ea-215698c88764" containerName="copy" Sep 10 12:48:08.489370 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:08.489302 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1274476-5f28-4d24-b7ea-215698c88764" containerName="gather" Sep 10 12:48:08.489370 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:08.489307 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1274476-5f28-4d24-b7ea-215698c88764" containerName="gather" Sep 10 12:48:08.489370 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:08.489364 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1274476-5f28-4d24-b7ea-215698c88764" containerName="copy" Sep 10 12:48:08.489535 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:08.489375 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1274476-5f28-4d24-b7ea-215698c88764" containerName="gather" Sep 10 12:48:08.492911 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:08.492891 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-5f526" Sep 10 12:48:08.495977 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:08.495957 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dxwh5\"/\"openshift-service-ca.crt\"" Sep 10 12:48:08.496085 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:08.495962 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dxwh5\"/\"kube-root-ca.crt\"" Sep 10 12:48:08.496807 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:08.496794 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dxwh5\"/\"default-dockercfg-dhffc\"" Sep 10 12:48:08.502128 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:08.502104 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dxwh5/perf-node-gather-daemonset-5f526"] Sep 10 12:48:08.649143 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:08.649098 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-br7zz\" (UniqueName: \"kubernetes.io/projected/6f0af5a9-ddff-40b0-9b6b-8eb75d2874cd-kube-api-access-br7zz\") pod \"perf-node-gather-daemonset-5f526\" (UID: \"6f0af5a9-ddff-40b0-9b6b-8eb75d2874cd\") " pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-5f526" Sep 10 12:48:08.649143 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:08.649160 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6f0af5a9-ddff-40b0-9b6b-8eb75d2874cd-lib-modules\") pod \"perf-node-gather-daemonset-5f526\" (UID: \"6f0af5a9-ddff-40b0-9b6b-8eb75d2874cd\") " pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-5f526" Sep 10 12:48:08.649387 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:08.649230 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f0af5a9-ddff-40b0-9b6b-8eb75d2874cd-sys\") pod \"perf-node-gather-daemonset-5f526\" (UID: \"6f0af5a9-ddff-40b0-9b6b-8eb75d2874cd\") " pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-5f526" Sep 10 12:48:08.649387 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:08.649268 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6f0af5a9-ddff-40b0-9b6b-8eb75d2874cd-podres\") pod \"perf-node-gather-daemonset-5f526\" (UID: \"6f0af5a9-ddff-40b0-9b6b-8eb75d2874cd\") " pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-5f526" Sep 10 12:48:08.649387 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:08.649308 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6f0af5a9-ddff-40b0-9b6b-8eb75d2874cd-proc\") pod \"perf-node-gather-daemonset-5f526\" (UID: \"6f0af5a9-ddff-40b0-9b6b-8eb75d2874cd\") " pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-5f526" Sep 10 12:48:08.749931 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:08.749810 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f0af5a9-ddff-40b0-9b6b-8eb75d2874cd-sys\") pod \"perf-node-gather-daemonset-5f526\" (UID: \"6f0af5a9-ddff-40b0-9b6b-8eb75d2874cd\") " pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-5f526" Sep 10 12:48:08.749931 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:08.749879 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6f0af5a9-ddff-40b0-9b6b-8eb75d2874cd-podres\") pod \"perf-node-gather-daemonset-5f526\" (UID: \"6f0af5a9-ddff-40b0-9b6b-8eb75d2874cd\") " pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-5f526" Sep 10 12:48:08.749931 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:08.749909 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6f0af5a9-ddff-40b0-9b6b-8eb75d2874cd-proc\") pod \"perf-node-gather-daemonset-5f526\" (UID: \"6f0af5a9-ddff-40b0-9b6b-8eb75d2874cd\") " pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-5f526" Sep 10 12:48:08.750191 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:08.749946 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f0af5a9-ddff-40b0-9b6b-8eb75d2874cd-sys\") pod \"perf-node-gather-daemonset-5f526\" (UID: \"6f0af5a9-ddff-40b0-9b6b-8eb75d2874cd\") " pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-5f526" Sep 10 12:48:08.750191 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:08.749981 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-br7zz\" (UniqueName: \"kubernetes.io/projected/6f0af5a9-ddff-40b0-9b6b-8eb75d2874cd-kube-api-access-br7zz\") pod \"perf-node-gather-daemonset-5f526\" (UID: \"6f0af5a9-ddff-40b0-9b6b-8eb75d2874cd\") " pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-5f526" Sep 10 12:48:08.750191 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:08.749983 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6f0af5a9-ddff-40b0-9b6b-8eb75d2874cd-podres\") pod \"perf-node-gather-daemonset-5f526\" (UID: \"6f0af5a9-ddff-40b0-9b6b-8eb75d2874cd\") " pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-5f526" Sep 10 12:48:08.750191 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:08.750016 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6f0af5a9-ddff-40b0-9b6b-8eb75d2874cd-proc\") pod \"perf-node-gather-daemonset-5f526\" (UID: \"6f0af5a9-ddff-40b0-9b6b-8eb75d2874cd\") " pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-5f526" Sep 10 12:48:08.750191 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:08.750032 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6f0af5a9-ddff-40b0-9b6b-8eb75d2874cd-lib-modules\") pod \"perf-node-gather-daemonset-5f526\" (UID: \"6f0af5a9-ddff-40b0-9b6b-8eb75d2874cd\") " pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-5f526" Sep 10 12:48:08.750191 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:08.750110 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6f0af5a9-ddff-40b0-9b6b-8eb75d2874cd-lib-modules\") pod \"perf-node-gather-daemonset-5f526\" (UID: \"6f0af5a9-ddff-40b0-9b6b-8eb75d2874cd\") " pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-5f526" Sep 10 12:48:08.758789 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:08.758761 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-br7zz\" (UniqueName: \"kubernetes.io/projected/6f0af5a9-ddff-40b0-9b6b-8eb75d2874cd-kube-api-access-br7zz\") pod \"perf-node-gather-daemonset-5f526\" (UID: \"6f0af5a9-ddff-40b0-9b6b-8eb75d2874cd\") " pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-5f526" Sep 10 12:48:08.802773 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:08.802742 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-5f526" Sep 10 12:48:09.071347 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:09.071199 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dxwh5/perf-node-gather-daemonset-5f526"] Sep 10 12:48:09.073382 ip-10-0-130-221 kubenswrapper[2580]: W0910 12:48:09.073356 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6f0af5a9_ddff_40b0_9b6b_8eb75d2874cd.slice/crio-671fe4390c58879a0219dafef972b378c25e7119f2b2279281aa3ad81f41e525 WatchSource:0}: Error finding container 671fe4390c58879a0219dafef972b378c25e7119f2b2279281aa3ad81f41e525: Status 404 returned error can't find the container with id 671fe4390c58879a0219dafef972b378c25e7119f2b2279281aa3ad81f41e525 Sep 10 12:48:09.253457 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:09.253426 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-5f526" event={"ID":"6f0af5a9-ddff-40b0-9b6b-8eb75d2874cd","Type":"ContainerStarted","Data":"90a834012da0713766d6a74c79ec65f2503ecdfe6850aade16152bdb50f23641"} Sep 10 12:48:09.253623 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:09.253472 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-5f526" event={"ID":"6f0af5a9-ddff-40b0-9b6b-8eb75d2874cd","Type":"ContainerStarted","Data":"671fe4390c58879a0219dafef972b378c25e7119f2b2279281aa3ad81f41e525"} Sep 10 12:48:09.253623 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:09.253537 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-5f526" Sep 10 12:48:09.275173 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:09.275128 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-5f526" podStartSLOduration=1.275114938 podStartE2EDuration="1.275114938s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:48:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:48:09.273699557 +0000 UTC m=+1809.281015939" watchObservedRunningTime="2026-09-10 12:48:09.275114938 +0000 UTC m=+1809.282431358" Sep 10 12:48:09.452472 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:09.452444 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sjwmh_42e548ce-35b0-4117-952f-c0b6bec98b14/dns/0.log" Sep 10 12:48:09.471123 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:09.471104 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sjwmh_42e548ce-35b0-4117-952f-c0b6bec98b14/kube-rbac-proxy/0.log" Sep 10 12:48:09.536905 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:09.536874 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pkt25_2a76158d-6412-48e2-a091-51cc34552ce2/dns-node-resolver/0.log" Sep 10 12:48:09.954647 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:09.954617 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-68mrh_e6b25f25-3e5d-45f3-8c46-7884a14b981f/node-ca/0.log" Sep 10 12:48:10.912739 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:10.912713 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2f4lc_e09b8c21-6ece-4589-8fa2-dfa2c0c0afe4/serve-healthcheck-canary/0.log" Sep 10 12:48:11.420034 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:11.420009 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t5mdr_5b1d6ed3-daa3-4642-84d3-dd8cfe53e110/kube-rbac-proxy/0.log" Sep 10 12:48:11.438237 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:11.438214 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t5mdr_5b1d6ed3-daa3-4642-84d3-dd8cfe53e110/exporter/0.log" Sep 10 12:48:11.457124 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:11.457094 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t5mdr_5b1d6ed3-daa3-4642-84d3-dd8cfe53e110/extractor/0.log" Sep 10 12:48:15.266476 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:15.266450 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-5f526" Sep 10 12:48:16.307131 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:16.307101 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-5g7lk_0c94a7b9-9724-416f-8b9e-b98acd8c3c41/kube-multus/0.log" Sep 10 12:48:16.472012 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:16.471987 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ft7lg_1d17b055-9484-48c5-bd77-90870f87d6ff/kube-multus-additional-cni-plugins/0.log" Sep 10 12:48:16.495848 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:16.495826 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ft7lg_1d17b055-9484-48c5-bd77-90870f87d6ff/egress-router-binary-copy/0.log" Sep 10 12:48:16.514713 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:16.514688 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ft7lg_1d17b055-9484-48c5-bd77-90870f87d6ff/cni-plugins/0.log" Sep 10 12:48:16.535380 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:16.535357 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ft7lg_1d17b055-9484-48c5-bd77-90870f87d6ff/bond-cni-plugin/0.log" Sep 10 12:48:16.556031 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:16.556007 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ft7lg_1d17b055-9484-48c5-bd77-90870f87d6ff/routeoverride-cni/0.log" Sep 10 12:48:16.574875 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:16.574803 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ft7lg_1d17b055-9484-48c5-bd77-90870f87d6ff/whereabouts-cni-bincopy/0.log" Sep 10 12:48:16.594080 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:16.594061 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ft7lg_1d17b055-9484-48c5-bd77-90870f87d6ff/whereabouts-cni/0.log" Sep 10 12:48:16.881550 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:16.881482 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7vx2g_7169f42e-6be8-4806-bf45-c77b6e2bdfd1/network-metrics-daemon/0.log" Sep 10 12:48:16.907868 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:16.907844 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7vx2g_7169f42e-6be8-4806-bf45-c77b6e2bdfd1/kube-rbac-proxy/0.log" Sep 10 12:48:18.154241 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:18.154216 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tfcmz_e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58/ovn-controller/0.log" Sep 10 12:48:18.171278 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:18.171250 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tfcmz_e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58/ovn-acl-logging/0.log" Sep 10 12:48:18.171278 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:18.171265 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tfcmz_e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58/ovn-acl-logging/1.log" Sep 10 12:48:18.187143 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:18.187118 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tfcmz_e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58/kube-rbac-proxy-node/0.log" Sep 10 12:48:18.205867 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:18.205840 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tfcmz_e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58/kube-rbac-proxy-ovn-metrics/0.log" Sep 10 12:48:18.224657 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:18.224634 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tfcmz_e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58/northd/0.log" Sep 10 12:48:18.244030 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:18.243956 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tfcmz_e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58/nbdb/0.log" Sep 10 12:48:18.262769 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:18.262747 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tfcmz_e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58/sbdb/0.log" Sep 10 12:48:18.296442 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:18.296415 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tfcmz_e828e8f0-b7fe-4d1c-850e-ffcc8ee4bd58/ovnkube-controller/0.log" Sep 10 12:48:19.299556 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:19.299479 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-d7ls5_1b89f971-1fee-46bf-ad4c-e68e6b83b9a0/network-check-target-container/0.log" Sep 10 12:48:20.135391 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:20.135362 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-ktr9w_6cda5c27-4386-49bd-8ad1-eb1e86535408/iptables-alerter/0.log" Sep 10 12:48:20.758840 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:20.758810 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-t8ths_c502cf69-0e72-4829-bb53-cd23f5367666/tuned/0.log" Sep 10 12:48:23.804390 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:23.804368 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-5mmp2_08a3068f-f5f3-4086-8e21-444a4df81149/csi-driver/0.log" Sep 10 12:48:23.823181 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:23.823161 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-5mmp2_08a3068f-f5f3-4086-8e21-444a4df81149/csi-node-driver-registrar/0.log" Sep 10 12:48:23.840781 ip-10-0-130-221 kubenswrapper[2580]: I0910 12:48:23.840751 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-5mmp2_08a3068f-f5f3-4086-8e21-444a4df81149/csi-liveness-probe/0.log"