Sep 08 08:09:46.551778 ip-10-0-143-1 systemd[1]: Starting Kubernetes Kubelet... Sep 08 08:09:46.935822 ip-10-0-143-1 kubenswrapper[2585]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 08 08:09:46.935822 ip-10-0-143-1 kubenswrapper[2585]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 08 08:09:46.935822 ip-10-0-143-1 kubenswrapper[2585]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 08 08:09:46.935822 ip-10-0-143-1 kubenswrapper[2585]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 08 08:09:46.935822 ip-10-0-143-1 kubenswrapper[2585]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 08 08:09:46.937016 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.936945 2585 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 08 08:09:46.940647 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940633 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 08:09:46.940647 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940647 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 08:09:46.940710 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940651 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 08:09:46.940710 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940655 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 08:09:46.940710 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940658 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 08:09:46.940710 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940661 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 08:09:46.940710 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940664 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 08:09:46.940710 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940673 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 08:09:46.940710 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940678 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 08:09:46.940710 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940682 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 08:09:46.940710 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940685 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 08:09:46.940710 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940687 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 08:09:46.940710 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940690 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 08:09:46.940710 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940692 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 08:09:46.940710 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940695 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 08:09:46.940710 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940697 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 08:09:46.940710 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940700 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 08:09:46.940710 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940702 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 08:09:46.940710 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940707 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 08:09:46.940710 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940710 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 08:09:46.940710 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940713 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 08:09:46.941168 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940716 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 08:09:46.941168 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940719 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 08:09:46.941168 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940722 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 08:09:46.941168 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940724 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 08:09:46.941168 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940727 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 08:09:46.941168 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940729 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 08:09:46.941168 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940732 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 08:09:46.941168 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940735 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 08:09:46.941168 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940737 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 08:09:46.941168 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940739 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 08:09:46.941168 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940742 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 08:09:46.941168 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940751 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 08:09:46.941168 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940754 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 08:09:46.941168 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940756 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 08:09:46.941168 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940759 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 08:09:46.941168 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940761 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 08:09:46.941168 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940764 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 08:09:46.941168 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940766 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 08:09:46.941168 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940768 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 08:09:46.941168 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940771 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 08:09:46.941715 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940773 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 08:09:46.941715 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940775 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 08:09:46.941715 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940779 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 08:09:46.941715 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940781 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 08:09:46.941715 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940784 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 08:09:46.941715 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940786 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 08:09:46.941715 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940788 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 08:09:46.941715 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940791 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 08:09:46.941715 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940794 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 08:09:46.941715 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940796 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 08:09:46.941715 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940799 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 08:09:46.941715 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940801 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 08:09:46.941715 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940804 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 08:09:46.941715 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940807 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 08:09:46.941715 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940809 2585 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 08:09:46.941715 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940812 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 08:09:46.941715 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940815 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 08:09:46.941715 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940817 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 08:09:46.941715 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940819 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 08:09:46.941715 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940822 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 08:09:46.942197 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940824 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 08:09:46.942197 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940827 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 08:09:46.942197 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940829 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 08:09:46.942197 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940831 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 08:09:46.942197 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940839 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 08:09:46.942197 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940841 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 08:09:46.942197 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940844 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 08:09:46.942197 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940846 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 08:09:46.942197 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940848 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 08:09:46.942197 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940851 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 08:09:46.942197 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940853 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 08:09:46.942197 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940855 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 08:09:46.942197 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940858 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 08:09:46.942197 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940860 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 08:09:46.942197 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940862 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 08:09:46.942197 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940865 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 08:09:46.942197 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940867 2585 feature_gate.go:328] unrecognized feature gate: Example Sep 08 08:09:46.942197 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940869 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 08:09:46.942197 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940872 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 08:09:46.942671 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940875 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 08:09:46.942671 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940881 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 08:09:46.942671 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940885 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 08:09:46.942671 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940889 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 08:09:46.942671 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940892 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 08:09:46.942671 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940895 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 08:09:46.942671 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940898 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 08:09:46.942671 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940900 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 08:09:46.942671 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940903 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 08:09:46.942671 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940906 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 08:09:46.942671 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940908 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 08:09:46.942671 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940911 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 08:09:46.942671 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940914 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 08:09:46.942671 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940916 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 08:09:46.942671 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940918 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 08:09:46.942671 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940921 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 08:09:46.942671 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940923 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 08:09:46.942671 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940925 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 08:09:46.942671 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.940928 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 08:09:46.942671 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941658 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 08:09:46.943167 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941665 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 08:09:46.943167 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941668 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 08:09:46.943167 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941672 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 08:09:46.943167 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941675 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 08:09:46.943167 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941678 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 08:09:46.943167 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941681 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 08:09:46.943167 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941684 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 08:09:46.943167 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941686 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 08:09:46.943167 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941689 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 08:09:46.943167 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941691 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 08:09:46.943167 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941694 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 08:09:46.943167 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941696 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 08:09:46.943167 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941699 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 08:09:46.943167 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941704 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 08:09:46.943167 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941708 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 08:09:46.943167 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941710 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 08:09:46.943167 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941714 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 08:09:46.943167 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941717 2585 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 08:09:46.943167 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941721 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 08:09:46.943648 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941724 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 08:09:46.943648 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941727 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 08:09:46.943648 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941730 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 08:09:46.943648 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941733 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 08:09:46.943648 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941735 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 08:09:46.943648 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941737 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 08:09:46.943648 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941740 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 08:09:46.943648 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941742 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 08:09:46.943648 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941745 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 08:09:46.943648 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941748 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 08:09:46.943648 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941750 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 08:09:46.943648 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941753 2585 feature_gate.go:328] unrecognized feature gate: Example Sep 08 08:09:46.943648 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941756 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 08:09:46.943648 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941759 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 08:09:46.943648 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941762 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 08:09:46.943648 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941765 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 08:09:46.943648 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941767 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 08:09:46.943648 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941770 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 08:09:46.943648 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941772 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 08:09:46.943648 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941775 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 08:09:46.944132 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941778 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 08:09:46.944132 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941781 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 08:09:46.944132 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941784 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 08:09:46.944132 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941786 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 08:09:46.944132 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941789 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 08:09:46.944132 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941791 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 08:09:46.944132 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941794 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 08:09:46.944132 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941797 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 08:09:46.944132 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941800 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 08:09:46.944132 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941802 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 08:09:46.944132 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941805 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 08:09:46.944132 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941808 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 08:09:46.944132 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941810 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 08:09:46.944132 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941813 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 08:09:46.944132 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941815 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 08:09:46.944132 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941818 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 08:09:46.944132 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941820 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 08:09:46.944132 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941823 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 08:09:46.944132 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941825 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 08:09:46.944132 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941828 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 08:09:46.944629 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941830 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 08:09:46.944629 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941834 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 08:09:46.944629 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941837 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 08:09:46.944629 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941840 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 08:09:46.944629 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941843 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 08:09:46.944629 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941846 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 08:09:46.944629 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941849 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 08:09:46.944629 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941852 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 08:09:46.944629 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941855 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 08:09:46.944629 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941857 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 08:09:46.944629 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941860 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 08:09:46.944629 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941862 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 08:09:46.944629 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941866 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 08:09:46.944629 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941868 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 08:09:46.944629 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941871 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 08:09:46.944629 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941873 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 08:09:46.944629 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941876 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 08:09:46.944629 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941878 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 08:09:46.944629 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941881 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 08:09:46.945105 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941884 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 08:09:46.945105 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941886 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 08:09:46.945105 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941889 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 08:09:46.945105 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941891 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 08:09:46.945105 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941894 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 08:09:46.945105 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941896 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 08:09:46.945105 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941899 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 08:09:46.945105 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941901 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 08:09:46.945105 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941903 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 08:09:46.945105 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941906 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 08:09:46.945105 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941908 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 08:09:46.945105 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941910 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 08:09:46.945105 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941913 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 08:09:46.945105 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941915 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 08:09:46.945105 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941918 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 08:09:46.945105 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941920 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 08:09:46.945105 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941922 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 08:09:46.945105 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941925 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 08:09:46.945105 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941927 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 08:09:46.945105 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.941930 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 08:09:46.945616 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942267 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 08:09:46.945616 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942272 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 08:09:46.945616 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942274 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 08:09:46.945616 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942277 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 08:09:46.945616 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942279 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 08:09:46.945616 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942282 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 08:09:46.945616 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942285 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 08:09:46.945616 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942288 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 08:09:46.945616 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942290 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 08:09:46.945616 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942293 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 08:09:46.945616 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942296 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 08:09:46.945616 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942298 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 08:09:46.945616 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942301 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 08:09:46.945616 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942303 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 08:09:46.945616 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942306 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 08:09:46.945616 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942308 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 08:09:46.945616 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942311 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 08:09:46.945616 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942314 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 08:09:46.945616 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942316 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 08:09:46.946071 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942319 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 08:09:46.946071 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942321 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 08:09:46.946071 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942324 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 08:09:46.946071 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942326 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 08:09:46.946071 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942328 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 08:09:46.946071 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942331 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 08:09:46.946071 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942333 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 08:09:46.946071 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942336 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 08:09:46.946071 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942338 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 08:09:46.946071 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942340 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 08:09:46.946071 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942343 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 08:09:46.946071 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942345 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 08:09:46.946071 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942348 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 08:09:46.946071 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942350 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 08:09:46.946071 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942353 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 08:09:46.946071 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942355 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 08:09:46.946071 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942357 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 08:09:46.946071 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942360 2585 feature_gate.go:328] unrecognized feature gate: Example Sep 08 08:09:46.946071 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942362 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 08:09:46.946071 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942365 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 08:09:46.946609 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942367 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 08:09:46.946609 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942370 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 08:09:46.946609 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942372 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 08:09:46.946609 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942390 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 08:09:46.946609 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942394 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 08:09:46.946609 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942396 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 08:09:46.946609 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942399 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 08:09:46.946609 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942402 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 08:09:46.946609 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942404 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 08:09:46.946609 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942406 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 08:09:46.946609 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942410 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 08:09:46.946609 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942414 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 08:09:46.946609 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942417 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 08:09:46.946609 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942419 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 08:09:46.946609 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942422 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 08:09:46.946609 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942424 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 08:09:46.946609 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942426 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 08:09:46.946609 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942429 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 08:09:46.946609 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942431 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 08:09:46.946609 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942433 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 08:09:46.947085 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942436 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 08:09:46.947085 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942438 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 08:09:46.947085 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942441 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 08:09:46.947085 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942443 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 08:09:46.947085 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942445 2585 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 08:09:46.947085 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942448 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 08:09:46.947085 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942451 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 08:09:46.947085 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942453 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 08:09:46.947085 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942456 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 08:09:46.947085 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942458 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 08:09:46.947085 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942460 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 08:09:46.947085 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942463 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 08:09:46.947085 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942469 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 08:09:46.947085 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942472 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 08:09:46.947085 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942474 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 08:09:46.947085 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942478 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 08:09:46.947085 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942482 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 08:09:46.947085 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942485 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 08:09:46.947085 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942488 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 08:09:46.947085 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942490 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 08:09:46.947584 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942493 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 08:09:46.947584 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942495 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 08:09:46.947584 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942498 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 08:09:46.947584 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942501 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 08:09:46.947584 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942503 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 08:09:46.947584 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942505 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 08:09:46.947584 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942508 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 08:09:46.947584 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942510 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 08:09:46.947584 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942513 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 08:09:46.947584 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942516 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 08:09:46.947584 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942518 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 08:09:46.947584 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942523 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 08:09:46.947584 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942526 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 08:09:46.947584 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942529 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 08:09:46.947584 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942531 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 08:09:46.947584 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942534 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 08:09:46.947584 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942537 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 08:09:46.947584 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942540 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 08:09:46.947584 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942543 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 08:09:46.948076 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.942545 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 08:09:46.948076 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942615 2585 flags.go:64] FLAG: --address="0.0.0.0" Sep 08 08:09:46.948076 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942649 2585 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 08 08:09:46.948076 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942657 2585 flags.go:64] FLAG: --anonymous-auth="true" Sep 08 08:09:46.948076 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942661 2585 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 08 08:09:46.948076 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942666 2585 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 08 08:09:46.948076 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942670 2585 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 08 08:09:46.948076 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942674 2585 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 08 08:09:46.948076 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942678 2585 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 08 08:09:46.948076 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942681 2585 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 08 08:09:46.948076 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942684 2585 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 08 08:09:46.948076 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942687 2585 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 08 08:09:46.948076 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942690 2585 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 08 08:09:46.948076 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942694 2585 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 08 08:09:46.948076 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942696 2585 flags.go:64] FLAG: --cgroup-root="" Sep 08 08:09:46.948076 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942699 2585 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 08 08:09:46.948076 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942702 2585 flags.go:64] FLAG: --client-ca-file="" Sep 08 08:09:46.948076 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942705 2585 flags.go:64] FLAG: --cloud-provider="external" Sep 08 08:09:46.948076 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942708 2585 flags.go:64] FLAG: --cluster-dns="[]" Sep 08 08:09:46.948076 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942712 2585 flags.go:64] FLAG: --cluster-domain="" Sep 08 08:09:46.948076 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942715 2585 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 08 08:09:46.948076 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942718 2585 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 08 08:09:46.948076 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942721 2585 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 08 08:09:46.948076 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942724 2585 flags.go:64] FLAG: --container-log-max-files="5" Sep 08 08:09:46.948716 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942727 2585 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 08 08:09:46.948716 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942730 2585 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 08 08:09:46.948716 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942733 2585 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 08 08:09:46.948716 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942736 2585 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 08 08:09:46.948716 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942739 2585 flags.go:64] FLAG: --contention-profiling="false" Sep 08 08:09:46.948716 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942742 2585 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 08 08:09:46.948716 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942745 2585 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 08 08:09:46.948716 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942748 2585 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 08 08:09:46.948716 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942751 2585 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 08 08:09:46.948716 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942755 2585 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 08 08:09:46.948716 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942758 2585 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 08 08:09:46.948716 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942762 2585 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 08 08:09:46.948716 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942765 2585 flags.go:64] FLAG: --enable-load-reader="false" Sep 08 08:09:46.948716 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942768 2585 flags.go:64] FLAG: --enable-server="true" Sep 08 08:09:46.948716 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942771 2585 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 08 08:09:46.948716 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942775 2585 flags.go:64] FLAG: --event-burst="100" Sep 08 08:09:46.948716 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942778 2585 flags.go:64] FLAG: --event-qps="50" Sep 08 08:09:46.948716 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942780 2585 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 08 08:09:46.948716 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942783 2585 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 08 08:09:46.948716 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942786 2585 flags.go:64] FLAG: --eviction-hard="" Sep 08 08:09:46.948716 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942789 2585 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 08 08:09:46.948716 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942792 2585 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 08 08:09:46.948716 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942795 2585 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 08 08:09:46.948716 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942798 2585 flags.go:64] FLAG: --eviction-soft="" Sep 08 08:09:46.948716 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942800 2585 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 08 08:09:46.949777 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942803 2585 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 08 08:09:46.949777 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942806 2585 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 08 08:09:46.949777 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942808 2585 flags.go:64] FLAG: --experimental-mounter-path="" Sep 08 08:09:46.949777 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942811 2585 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 08 08:09:46.949777 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942814 2585 flags.go:64] FLAG: --fail-swap-on="true" Sep 08 08:09:46.949777 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942816 2585 flags.go:64] FLAG: --feature-gates="" Sep 08 08:09:46.949777 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942820 2585 flags.go:64] FLAG: --file-check-frequency="20s" Sep 08 08:09:46.949777 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942822 2585 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 08 08:09:46.949777 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942825 2585 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 08 08:09:46.949777 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942828 2585 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 08 08:09:46.949777 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942831 2585 flags.go:64] FLAG: --healthz-port="10248" Sep 08 08:09:46.949777 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942833 2585 flags.go:64] FLAG: --help="false" Sep 08 08:09:46.949777 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942836 2585 flags.go:64] FLAG: --hostname-override="ip-10-0-143-1.ec2.internal" Sep 08 08:09:46.949777 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942839 2585 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 08 08:09:46.949777 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942841 2585 flags.go:64] FLAG: --http-check-frequency="20s" Sep 08 08:09:46.949777 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942844 2585 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 08 08:09:46.949777 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942848 2585 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 08 08:09:46.949777 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942851 2585 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 08 08:09:46.949777 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942855 2585 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 08 08:09:46.949777 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942858 2585 flags.go:64] FLAG: --image-service-endpoint="" Sep 08 08:09:46.949777 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942861 2585 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 08 08:09:46.949777 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942864 2585 flags.go:64] FLAG: --kube-api-burst="100" Sep 08 08:09:46.949777 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942866 2585 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 08 08:09:46.950712 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942869 2585 flags.go:64] FLAG: --kube-api-qps="50" Sep 08 08:09:46.950712 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942872 2585 flags.go:64] FLAG: --kube-reserved="" Sep 08 08:09:46.950712 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942875 2585 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 08 08:09:46.950712 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942878 2585 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 08 08:09:46.950712 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942880 2585 flags.go:64] FLAG: --kubelet-cgroups="" Sep 08 08:09:46.950712 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942883 2585 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 08 08:09:46.950712 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942886 2585 flags.go:64] FLAG: --lock-file="" Sep 08 08:09:46.950712 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942888 2585 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 08 08:09:46.950712 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942891 2585 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 08 08:09:46.950712 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942894 2585 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 08 08:09:46.950712 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942899 2585 flags.go:64] FLAG: --log-json-split-stream="false" Sep 08 08:09:46.950712 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942902 2585 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 08 08:09:46.950712 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942904 2585 flags.go:64] FLAG: --log-text-split-stream="false" Sep 08 08:09:46.950712 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942907 2585 flags.go:64] FLAG: --logging-format="text" Sep 08 08:09:46.950712 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942910 2585 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 08 08:09:46.950712 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942913 2585 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 08 08:09:46.950712 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942915 2585 flags.go:64] FLAG: --manifest-url="" Sep 08 08:09:46.950712 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942918 2585 flags.go:64] FLAG: --manifest-url-header="" Sep 08 08:09:46.950712 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942921 2585 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 08 08:09:46.950712 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942924 2585 flags.go:64] FLAG: --max-open-files="1000000" Sep 08 08:09:46.950712 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942928 2585 flags.go:64] FLAG: --max-pods="110" Sep 08 08:09:46.950712 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942931 2585 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 08 08:09:46.950712 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942933 2585 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 08 08:09:46.950712 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942936 2585 flags.go:64] FLAG: --memory-manager-policy="None" Sep 08 08:09:46.950712 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942939 2585 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 08 08:09:46.951344 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942941 2585 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 08 08:09:46.951344 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942944 2585 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 08 08:09:46.951344 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942949 2585 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 08 08:09:46.951344 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942955 2585 flags.go:64] FLAG: --node-status-max-images="50" Sep 08 08:09:46.951344 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942958 2585 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 08 08:09:46.951344 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942961 2585 flags.go:64] FLAG: --oom-score-adj="-999" Sep 08 08:09:46.951344 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942964 2585 flags.go:64] FLAG: --pod-cidr="" Sep 08 08:09:46.951344 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942967 2585 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 08 08:09:46.951344 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942972 2585 flags.go:64] FLAG: --pod-manifest-path="" Sep 08 08:09:46.951344 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942975 2585 flags.go:64] FLAG: --pod-max-pids="-1" Sep 08 08:09:46.951344 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942978 2585 flags.go:64] FLAG: --pods-per-core="0" Sep 08 08:09:46.951344 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942980 2585 flags.go:64] FLAG: --port="10250" Sep 08 08:09:46.951344 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942983 2585 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 08 08:09:46.951344 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942986 2585 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-05cde8d4f4fe645c5" Sep 08 08:09:46.951344 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942989 2585 flags.go:64] FLAG: --qos-reserved="" Sep 08 08:09:46.951344 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942992 2585 flags.go:64] FLAG: --read-only-port="10255" Sep 08 08:09:46.951344 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942995 2585 flags.go:64] FLAG: --register-node="true" Sep 08 08:09:46.951344 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.942997 2585 flags.go:64] FLAG: --register-with-taints="" Sep 08 08:09:46.951344 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.943001 2585 flags.go:64] FLAG: --registry-burst="10" Sep 08 08:09:46.951344 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.943004 2585 flags.go:64] FLAG: --registry-qps="5" Sep 08 08:09:46.951344 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.943006 2585 flags.go:64] FLAG: --reserved-cpus="" Sep 08 08:09:46.951344 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.943009 2585 flags.go:64] FLAG: --reserved-memory="" Sep 08 08:09:46.951344 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.943012 2585 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 08 08:09:46.951344 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.943015 2585 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 08 08:09:46.951344 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.943018 2585 flags.go:64] FLAG: --rotate-certificates="false" Sep 08 08:09:46.952030 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.943021 2585 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 08 08:09:46.952030 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.943023 2585 flags.go:64] FLAG: --runonce="false" Sep 08 08:09:46.952030 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.943026 2585 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 08 08:09:46.952030 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.943029 2585 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 08 08:09:46.952030 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.943032 2585 flags.go:64] FLAG: --seccomp-default="false" Sep 08 08:09:46.952030 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.943034 2585 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 08 08:09:46.952030 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.943037 2585 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 08 08:09:46.952030 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.943039 2585 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 08 08:09:46.952030 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.943054 2585 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 08 08:09:46.952030 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.943060 2585 flags.go:64] FLAG: --storage-driver-password="root" Sep 08 08:09:46.952030 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.943062 2585 flags.go:64] FLAG: --storage-driver-secure="false" Sep 08 08:09:46.952030 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.943065 2585 flags.go:64] FLAG: --storage-driver-table="stats" Sep 08 08:09:46.952030 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.943068 2585 flags.go:64] FLAG: --storage-driver-user="root" Sep 08 08:09:46.952030 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.943070 2585 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 08 08:09:46.952030 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.943073 2585 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 08 08:09:46.952030 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.943076 2585 flags.go:64] FLAG: --system-cgroups="" Sep 08 08:09:46.952030 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.943079 2585 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 08 08:09:46.952030 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.943084 2585 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 08 08:09:46.952030 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.943086 2585 flags.go:64] FLAG: --tls-cert-file="" Sep 08 08:09:46.952030 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.943089 2585 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 08 08:09:46.952030 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.943093 2585 flags.go:64] FLAG: --tls-min-version="" Sep 08 08:09:46.952030 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.943096 2585 flags.go:64] FLAG: --tls-private-key-file="" Sep 08 08:09:46.952030 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.943099 2585 flags.go:64] FLAG: --topology-manager-policy="none" Sep 08 08:09:46.952030 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.943101 2585 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 08 08:09:46.952030 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.943104 2585 flags.go:64] FLAG: --topology-manager-scope="container" Sep 08 08:09:46.952700 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.943107 2585 flags.go:64] FLAG: --v="2" Sep 08 08:09:46.952700 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.943111 2585 flags.go:64] FLAG: --version="false" Sep 08 08:09:46.952700 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.943115 2585 flags.go:64] FLAG: --vmodule="" Sep 08 08:09:46.952700 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.943118 2585 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 08 08:09:46.952700 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.943121 2585 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 08 08:09:46.952700 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943228 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 08:09:46.952700 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943232 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 08:09:46.952700 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943234 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 08:09:46.952700 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943238 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 08:09:46.952700 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943242 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 08:09:46.952700 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943245 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 08:09:46.952700 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943247 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 08:09:46.952700 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943249 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 08:09:46.952700 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943252 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 08:09:46.952700 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943256 2585 feature_gate.go:328] unrecognized feature gate: Example Sep 08 08:09:46.952700 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943258 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 08:09:46.952700 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943262 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 08:09:46.952700 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943266 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 08:09:46.952700 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943269 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 08:09:46.952700 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943272 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 08:09:46.952700 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943275 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 08:09:46.953838 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943278 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 08:09:46.953838 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943282 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 08:09:46.953838 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943285 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 08:09:46.953838 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943288 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 08:09:46.953838 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943290 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 08:09:46.953838 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943293 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 08:09:46.953838 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943296 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 08:09:46.953838 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943298 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 08:09:46.953838 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943301 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 08:09:46.953838 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943303 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 08:09:46.953838 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943306 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 08:09:46.953838 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943308 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 08:09:46.953838 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943311 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 08:09:46.953838 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943313 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 08:09:46.953838 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943316 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 08:09:46.953838 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943319 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 08:09:46.953838 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943321 2585 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 08:09:46.953838 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943324 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 08:09:46.953838 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943326 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 08:09:46.954301 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943328 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 08:09:46.954301 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943331 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 08:09:46.954301 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943333 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 08:09:46.954301 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943336 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 08:09:46.954301 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943338 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 08:09:46.954301 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943340 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 08:09:46.954301 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943344 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 08:09:46.954301 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943347 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 08:09:46.954301 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943351 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 08:09:46.954301 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943353 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 08:09:46.954301 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943356 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 08:09:46.954301 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943358 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 08:09:46.954301 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943361 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 08:09:46.954301 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943363 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 08:09:46.954301 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943366 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 08:09:46.954301 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943368 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 08:09:46.954301 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943371 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 08:09:46.954301 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943373 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 08:09:46.954301 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943388 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 08:09:46.954301 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943391 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 08:09:46.954800 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943394 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 08:09:46.954800 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943397 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 08:09:46.954800 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943399 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 08:09:46.954800 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943401 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 08:09:46.954800 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943404 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 08:09:46.954800 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943406 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 08:09:46.954800 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943409 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 08:09:46.954800 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943411 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 08:09:46.954800 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943414 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 08:09:46.954800 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943416 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 08:09:46.954800 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943419 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 08:09:46.954800 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943421 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 08:09:46.954800 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943424 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 08:09:46.954800 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943426 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 08:09:46.954800 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943429 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 08:09:46.954800 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943431 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 08:09:46.954800 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943433 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 08:09:46.954800 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943436 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 08:09:46.954800 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943439 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 08:09:46.954800 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943442 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 08:09:46.955406 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943445 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 08:09:46.955406 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943448 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 08:09:46.955406 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943450 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 08:09:46.955406 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943453 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 08:09:46.955406 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943456 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 08:09:46.955406 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943458 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 08:09:46.955406 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943461 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 08:09:46.955406 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943463 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 08:09:46.955406 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943466 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 08:09:46.955406 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943468 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 08:09:46.955406 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943471 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 08:09:46.955406 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943473 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 08:09:46.955406 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943476 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 08:09:46.955406 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943478 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 08:09:46.955406 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943481 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 08:09:46.955406 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943483 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 08:09:46.955406 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943485 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 08:09:46.955406 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943488 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 08:09:46.955406 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943490 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 08:09:46.955406 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943493 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 08:09:46.955908 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943495 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 08:09:46.955908 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943497 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 08:09:46.955908 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943500 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 08:09:46.955908 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.943502 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 08:09:46.955908 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.943507 2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 08 08:09:46.955908 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.951976 2585 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 08 08:09:46.955908 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.952090 2585 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 08 08:09:46.955908 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.952925 2585 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 08 08:09:46.955908 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.952935 2585 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 08 08:09:46.955908 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.952994 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 08:09:46.955908 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.952999 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 08:09:46.955908 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953002 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 08:09:46.955908 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953005 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 08:09:46.955908 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953008 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 08:09:46.955908 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953010 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 08:09:46.956287 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953013 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 08:09:46.956287 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953015 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 08:09:46.956287 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953018 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 08:09:46.956287 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953021 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 08:09:46.956287 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953023 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 08:09:46.956287 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953026 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 08:09:46.956287 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953029 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 08:09:46.956287 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953032 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 08:09:46.956287 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953034 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 08:09:46.956287 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953036 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 08:09:46.956287 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953039 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 08:09:46.956287 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953042 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 08:09:46.956287 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953045 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 08:09:46.956287 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953048 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 08:09:46.956287 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953050 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 08:09:46.956287 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953053 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 08:09:46.956287 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953055 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 08:09:46.956287 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953058 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 08:09:46.956287 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953062 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 08:09:46.956287 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953065 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 08:09:46.956814 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953068 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 08:09:46.956814 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953071 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 08:09:46.956814 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953075 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 08:09:46.956814 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953078 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 08:09:46.956814 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953081 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 08:09:46.956814 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953083 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 08:09:46.956814 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953086 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 08:09:46.956814 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953088 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 08:09:46.956814 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953091 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 08:09:46.956814 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953094 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 08:09:46.956814 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953097 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 08:09:46.956814 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953099 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 08:09:46.956814 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953102 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 08:09:46.956814 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953104 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 08:09:46.956814 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953107 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 08:09:46.956814 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953109 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 08:09:46.956814 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953112 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 08:09:46.956814 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953114 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 08:09:46.956814 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953118 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 08:09:46.956814 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953121 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 08:09:46.957302 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953124 2585 feature_gate.go:328] unrecognized feature gate: Example Sep 08 08:09:46.957302 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953126 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 08:09:46.957302 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953129 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 08:09:46.957302 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953131 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 08:09:46.957302 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953134 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 08:09:46.957302 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953136 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 08:09:46.957302 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953139 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 08:09:46.957302 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953141 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 08:09:46.957302 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953144 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 08:09:46.957302 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953146 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 08:09:46.957302 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953149 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 08:09:46.957302 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953151 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 08:09:46.957302 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953153 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 08:09:46.957302 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953156 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 08:09:46.957302 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953158 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 08:09:46.957302 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953162 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 08:09:46.957302 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953166 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 08:09:46.957302 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953170 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 08:09:46.957302 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953172 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 08:09:46.957848 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953175 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 08:09:46.957848 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953178 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 08:09:46.957848 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953180 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 08:09:46.957848 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953183 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 08:09:46.957848 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953185 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 08:09:46.957848 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953188 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 08:09:46.957848 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953190 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 08:09:46.957848 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953192 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 08:09:46.957848 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953195 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 08:09:46.957848 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953197 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 08:09:46.957848 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953199 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 08:09:46.957848 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953202 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 08:09:46.957848 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953204 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 08:09:46.957848 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953207 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 08:09:46.957848 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953210 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 08:09:46.957848 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953213 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 08:09:46.957848 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953215 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 08:09:46.957848 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953218 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 08:09:46.957848 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953220 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 08:09:46.957848 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953223 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 08:09:46.958327 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953226 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 08:09:46.958327 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953228 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 08:09:46.958327 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953231 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 08:09:46.958327 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953233 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 08:09:46.958327 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953236 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 08:09:46.958327 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953238 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 08:09:46.958327 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953240 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 08:09:46.958327 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953243 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 08:09:46.958327 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953245 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 08:09:46.958327 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953248 2585 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 08:09:46.958327 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953250 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 08:09:46.958327 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953253 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 08:09:46.958327 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953255 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 08:09:46.958327 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953257 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 08:09:46.958327 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.953262 2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 08 08:09:46.958708 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953412 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 08:09:46.958708 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953419 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 08:09:46.958708 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953421 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 08:09:46.958708 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953424 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 08:09:46.958708 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953427 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 08:09:46.958708 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953429 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 08:09:46.958708 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953432 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 08:09:46.958708 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953434 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 08:09:46.958708 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953437 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 08:09:46.958708 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953439 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 08:09:46.958708 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953441 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 08:09:46.958708 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953444 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 08:09:46.958708 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953447 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 08:09:46.958708 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953450 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 08:09:46.958708 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953452 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 08:09:46.958708 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953455 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 08:09:46.958708 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953458 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 08:09:46.958708 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953460 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 08:09:46.958708 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953463 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 08:09:46.958708 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953466 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 08:09:46.959203 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953468 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 08:09:46.959203 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953471 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 08:09:46.959203 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953473 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 08:09:46.959203 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953475 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 08:09:46.959203 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953478 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 08:09:46.959203 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953480 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 08:09:46.959203 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953483 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 08:09:46.959203 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953486 2585 feature_gate.go:328] unrecognized feature gate: Example Sep 08 08:09:46.959203 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953488 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 08:09:46.959203 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953493 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 08:09:46.959203 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953497 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 08:09:46.959203 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953499 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 08:09:46.959203 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953503 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 08:09:46.959203 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953507 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 08:09:46.959203 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953509 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 08:09:46.959203 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953512 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 08:09:46.959203 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953515 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 08:09:46.959203 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953518 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 08:09:46.959203 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953521 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 08:09:46.959670 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953524 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 08:09:46.959670 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953527 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 08:09:46.959670 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953529 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 08:09:46.959670 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953531 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 08:09:46.959670 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953534 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 08:09:46.959670 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953536 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 08:09:46.959670 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953546 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 08:09:46.959670 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953548 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 08:09:46.959670 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953551 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 08:09:46.959670 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953554 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 08:09:46.959670 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953557 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 08:09:46.959670 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953560 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 08:09:46.959670 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953562 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 08:09:46.959670 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953564 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 08:09:46.959670 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953567 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 08:09:46.959670 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953569 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 08:09:46.959670 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953572 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 08:09:46.959670 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953574 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 08:09:46.959670 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953577 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 08:09:46.959670 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953579 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 08:09:46.960137 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953582 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 08:09:46.960137 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953584 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 08:09:46.960137 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953587 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 08:09:46.960137 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953589 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 08:09:46.960137 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953592 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 08:09:46.960137 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953594 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 08:09:46.960137 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953596 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 08:09:46.960137 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953599 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 08:09:46.960137 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953602 2585 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 08:09:46.960137 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953604 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 08:09:46.960137 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953606 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 08:09:46.960137 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953609 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 08:09:46.960137 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953611 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 08:09:46.960137 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953614 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 08:09:46.960137 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953616 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 08:09:46.960137 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953618 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 08:09:46.960137 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953621 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 08:09:46.960137 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953623 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 08:09:46.960137 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953626 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 08:09:46.960607 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953629 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 08:09:46.960607 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953632 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 08:09:46.960607 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953634 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 08:09:46.960607 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953637 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 08:09:46.960607 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953639 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 08:09:46.960607 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953641 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 08:09:46.960607 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953644 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 08:09:46.960607 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953646 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 08:09:46.960607 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953648 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 08:09:46.960607 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953651 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 08:09:46.960607 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953653 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 08:09:46.960607 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953655 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 08:09:46.960607 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953658 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 08:09:46.960607 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953661 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 08:09:46.960607 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953663 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 08:09:46.960607 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953666 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 08:09:46.960607 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953668 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 08:09:46.960607 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953670 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 08:09:46.960607 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953673 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 08:09:46.961056 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953676 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 08:09:46.961056 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:46.953680 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 08:09:46.961056 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.953684 2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 08 08:09:46.961056 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.953783 2585 server.go:962] "Client rotation is on, will bootstrap in background" Sep 08 08:09:46.961056 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.955856 2585 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 08 08:09:46.961056 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.956842 2585 server.go:1019] "Starting client certificate rotation" Sep 08 08:09:46.961056 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.956944 2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 08 08:09:46.961056 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.956977 2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 08 08:09:46.961056 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.959148 2585 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 08 08:09:46.962804 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.962791 2585 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 08 08:09:46.973718 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.973698 2585 log.go:25] "Validated CRI v1 runtime API" Sep 08 08:09:46.976819 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.976800 2585 log.go:25] "Validated CRI v1 image API" Sep 08 08:09:46.977696 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.977681 2585 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 08 08:09:46.981649 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.981630 2585 fs.go:135] Filesystem UUIDs: map[1f09091b-efbf-4ce2-b7ee-538b6aabf8b1:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 f0ddda4c-a11f-4147-811c-6a3b3b9d3298:/dev/nvme0n1p4] Sep 08 08:09:46.981711 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.981648 2585 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 08 08:09:46.983563 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.983548 2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 08 08:09:46.987632 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.987532 2585 manager.go:216] Machine: {Timestamp:2026-09-08 08:09:46.986290931 +0000 UTC m=+0.323930562 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099423 MemoryCapacity:32812163072 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2208562f40cdd1255c1db106c5eefa SystemUUID:ec220856-2f40-cdd1-255c-1db106c5eefa BootID:96f5a934-1e3f-4ff0-be65-67a139dbac7f Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:af:78:81:f8:0f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:af:78:81:f8:0f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9e:0b:52:ae:04:eb Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812163072 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 08 08:09:46.987632 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.987630 2585 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 08 08:09:46.987735 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.987722 2585 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 08 08:09:46.990079 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.990054 2585 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 08 08:09:46.990201 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.990082 2585 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-1.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 08 08:09:46.990821 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.990804 2585 topology_manager.go:138] "Creating topology manager with none policy" Sep 08 08:09:46.991182 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.990823 2585 container_manager_linux.go:309] "Creating device plugin manager" Sep 08 08:09:46.991215 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.991185 2585 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 08 08:09:46.992508 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.992498 2585 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 08 08:09:46.993280 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.993272 2585 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 08 08:09:46.996071 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.996060 2585 state_mem.go:36] "Initialized new in-memory state store" Sep 08 08:09:46.996311 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.996301 2585 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 08 08:09:46.996343 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.996332 2585 kubelet.go:486] "Attempting to sync node with API server" Sep 08 08:09:46.996371 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.996343 2585 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 08 08:09:46.996371 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.996354 2585 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 08 08:09:46.996371 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.996362 2585 kubelet.go:398] "Adding apiserver pod source" Sep 08 08:09:46.996371 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.996370 2585 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 08 08:09:46.998034 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.998005 2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 08 08:09:46.998131 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:46.998059 2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 08 08:09:47.000565 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.000547 2585 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 08 08:09:47.003016 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.003002 2585 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 08 08:09:47.003078 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.003033 2585 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 08 08:09:47.004069 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.004059 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 08 08:09:47.004109 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.004074 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 08 08:09:47.004109 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.004081 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 08 08:09:47.004109 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.004086 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 08 08:09:47.004109 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.004094 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 08 08:09:47.004109 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.004100 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 08 08:09:47.004232 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.004113 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 08 08:09:47.004232 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.004118 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 08 08:09:47.004232 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.004125 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 08 08:09:47.004232 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.004130 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 08 08:09:47.004232 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.004138 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 08 08:09:47.004232 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.004142 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 08 08:09:47.005371 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.005360 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 08 08:09:47.005371 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.005370 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 08 08:09:47.007566 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:47.007545 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 08 08:09:47.007566 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:47.007548 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-1.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 08 08:09:47.007665 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.007633 2585 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-1.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 08 08:09:47.008860 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.008845 2585 server.go:1268] "Started kubelet" Sep 08 08:09:47.008949 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.008923 2585 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 08 08:09:47.009006 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.008935 2585 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 08 08:09:47.009006 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.009002 2585 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 08 08:09:47.009521 ip-10-0-143-1 systemd[1]: Started Kubernetes Kubelet. Sep 08 08:09:47.009889 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.009870 2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2vzfg" Sep 08 08:09:47.010102 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.010073 2585 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 08 08:09:47.011295 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.011282 2585 server.go:310] "Adding debug handlers to kubelet server" Sep 08 08:09:47.014136 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:47.014118 2585 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 08 08:09:47.014963 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.014945 2585 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 08 08:09:47.015061 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.015007 2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 08 08:09:47.015268 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:47.015230 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-1.ec2.internal\" not found" Sep 08 08:09:47.015626 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.015609 2585 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 08 08:09:47.015748 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.015736 2585 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 08 08:09:47.015874 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.015843 2585 factory.go:55] Registering systemd factory Sep 08 08:09:47.015874 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.015609 2585 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 08 08:09:47.015874 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.015866 2585 factory.go:223] Registration of the systemd container factory successfully Sep 08 08:09:47.016078 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.015910 2585 reconstruct.go:97] "Volume reconstruction finished" Sep 08 08:09:47.016078 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.015916 2585 reconciler.go:29] "Reconciler: start to sync state" Sep 08 08:09:47.016279 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.016251 2585 factory.go:153] Registering CRI-O factory Sep 08 08:09:47.016279 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.016271 2585 factory.go:223] Registration of the crio container factory successfully Sep 08 08:09:47.016445 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.016331 2585 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 08 08:09:47.016445 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.016349 2585 factory.go:103] Registering Raw factory Sep 08 08:09:47.016445 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.016362 2585 manager.go:1196] Started watching for new ooms in manager Sep 08 08:09:47.016686 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.016672 2585 manager.go:349] Starting recovery of all containers Sep 08 08:09:47.017824 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.017796 2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2vzfg" Sep 08 08:09:47.024794 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:47.021307 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 08 08:09:47.024794 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:47.021406 2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-1.ec2.internal.18d34a220758bddd default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-1.ec2.internal,UID:ip-10-0-143-1.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-1.ec2.internal,},FirstTimestamp:2026-09-08 08:09:47.008818653 +0000 UTC m=+0.346458284,LastTimestamp:2026-09-08 08:09:47.008818653 +0000 UTC m=+0.346458284,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-1.ec2.internal,}" Sep 08 08:09:47.026121 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:47.026095 2585 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-1.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 08 08:09:47.027665 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.027648 2585 manager.go:354] Recovery completed Sep 08 08:09:47.032927 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.032912 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 08:09:47.034638 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.034622 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-1.ec2.internal" event="NodeHasSufficientMemory" Sep 08 08:09:47.034701 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.034649 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-1.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 08:09:47.034701 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.034659 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-1.ec2.internal" event="NodeHasSufficientPID" Sep 08 08:09:47.035132 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.035117 2585 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 08 08:09:47.035193 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.035132 2585 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 08 08:09:47.035193 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.035146 2585 state_mem.go:36] "Initialized new in-memory state store" Sep 08 08:09:47.035193 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.035171 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 08:09:47.035833 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.035820 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-1.ec2.internal" event="NodeHasSufficientMemory" Sep 08 08:09:47.035897 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.035845 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-1.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 08:09:47.035897 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.035854 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-1.ec2.internal" event="NodeHasSufficientPID" Sep 08 08:09:47.036197 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.036186 2585 policy_none.go:49] "None policy: Start" Sep 08 08:09:47.036237 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.036202 2585 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 08 08:09:47.036237 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.036212 2585 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 08 08:09:47.037206 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.037196 2585 policy_none.go:47] "Start" Sep 08 08:09:47.071968 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.071912 2585 manager.go:337] "Starting Device Plugin manager" Sep 08 08:09:47.071968 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:47.071954 2585 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 08 08:09:47.071968 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.071966 2585 server.go:87] "Starting device plugin registration server" Sep 08 08:09:47.072203 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.072186 2585 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 08 08:09:47.072281 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.072203 2585 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 08 08:09:47.072412 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.072398 2585 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 08 08:09:47.072475 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.072467 2585 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 08 08:09:47.072475 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.072476 2585 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 08 08:09:47.073371 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:47.073347 2585 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 08 08:09:47.073605 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:47.073426 2585 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-1.ec2.internal\" not found" Sep 08 08:09:47.167509 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.167471 2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 08 08:09:47.168739 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.168725 2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 08 08:09:47.168820 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.168777 2585 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 08 08:09:47.168820 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.168808 2585 kubelet.go:2448] "Starting kubelet main sync loop" Sep 08 08:09:47.168913 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:47.168853 2585 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 08 08:09:47.171056 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.171035 2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 08:09:47.172641 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.172626 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 08:09:47.173344 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.173318 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-1.ec2.internal" event="NodeHasSufficientMemory" Sep 08 08:09:47.173446 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.173355 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-1.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 08:09:47.173446 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.173365 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-1.ec2.internal" event="NodeHasSufficientPID" Sep 08 08:09:47.173446 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.173401 2585 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-1.ec2.internal" Sep 08 08:09:47.182278 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.182263 2585 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-1.ec2.internal" Sep 08 08:09:47.182332 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:47.182283 2585 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-1.ec2.internal\": node \"ip-10-0-143-1.ec2.internal\" not found" Sep 08 08:09:47.196589 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:47.196535 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-1.ec2.internal\" not found" Sep 08 08:09:47.269439 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.269416 2585 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-1.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-1.ec2.internal"] Sep 08 08:09:47.269518 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.269499 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 08:09:47.270894 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.270874 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-1.ec2.internal" event="NodeHasSufficientMemory" Sep 08 08:09:47.270964 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.270905 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-1.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 08:09:47.270964 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.270917 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-1.ec2.internal" event="NodeHasSufficientPID" Sep 08 08:09:47.271973 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.271962 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 08:09:47.272075 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.272063 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-1.ec2.internal" Sep 08 08:09:47.272108 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.272088 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 08:09:47.272698 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.272676 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-1.ec2.internal" event="NodeHasSufficientMemory" Sep 08 08:09:47.272698 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.272694 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-1.ec2.internal" event="NodeHasSufficientMemory" Sep 08 08:09:47.272855 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.272704 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-1.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 08:09:47.272855 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.272716 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-1.ec2.internal" event="NodeHasSufficientPID" Sep 08 08:09:47.272855 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.272716 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-1.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 08:09:47.272957 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.272860 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-1.ec2.internal" event="NodeHasSufficientPID" Sep 08 08:09:47.273824 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.273811 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-1.ec2.internal" Sep 08 08:09:47.273867 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.273834 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 08:09:47.275550 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.275536 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-1.ec2.internal" event="NodeHasSufficientMemory" Sep 08 08:09:47.275599 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.275556 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-1.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 08:09:47.275599 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.275573 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-1.ec2.internal" event="NodeHasSufficientPID" Sep 08 08:09:47.297021 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:47.297004 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-1.ec2.internal\" not found" Sep 08 08:09:47.299901 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:47.299888 2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-1.ec2.internal\" not found" node="ip-10-0-143-1.ec2.internal" Sep 08 08:09:47.303113 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:47.303099 2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-1.ec2.internal\" not found" node="ip-10-0-143-1.ec2.internal" Sep 08 08:09:47.324252 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.324233 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/67c6381f04db5b84e9ecdacab28bc19f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-1.ec2.internal\" (UID: \"67c6381f04db5b84e9ecdacab28bc19f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-1.ec2.internal" Sep 08 08:09:47.324308 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.324257 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/67c6381f04db5b84e9ecdacab28bc19f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-1.ec2.internal\" (UID: \"67c6381f04db5b84e9ecdacab28bc19f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-1.ec2.internal" Sep 08 08:09:47.324308 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.324275 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6f38569e02cf6895d157f6aa28cad740-config\") pod \"kube-apiserver-proxy-ip-10-0-143-1.ec2.internal\" (UID: \"6f38569e02cf6895d157f6aa28cad740\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-1.ec2.internal" Sep 08 08:09:47.397934 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:47.397914 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-1.ec2.internal\" not found" Sep 08 08:09:47.425260 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.425241 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/67c6381f04db5b84e9ecdacab28bc19f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-1.ec2.internal\" (UID: \"67c6381f04db5b84e9ecdacab28bc19f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-1.ec2.internal" Sep 08 08:09:47.425317 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.425263 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/67c6381f04db5b84e9ecdacab28bc19f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-1.ec2.internal\" (UID: \"67c6381f04db5b84e9ecdacab28bc19f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-1.ec2.internal" Sep 08 08:09:47.425317 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.425279 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6f38569e02cf6895d157f6aa28cad740-config\") pod \"kube-apiserver-proxy-ip-10-0-143-1.ec2.internal\" (UID: \"6f38569e02cf6895d157f6aa28cad740\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-1.ec2.internal" Sep 08 08:09:47.425317 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.425305 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6f38569e02cf6895d157f6aa28cad740-config\") pod \"kube-apiserver-proxy-ip-10-0-143-1.ec2.internal\" (UID: \"6f38569e02cf6895d157f6aa28cad740\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-1.ec2.internal" Sep 08 08:09:47.425450 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.425323 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/67c6381f04db5b84e9ecdacab28bc19f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-1.ec2.internal\" (UID: \"67c6381f04db5b84e9ecdacab28bc19f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-1.ec2.internal" Sep 08 08:09:47.425450 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.425342 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/67c6381f04db5b84e9ecdacab28bc19f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-1.ec2.internal\" (UID: \"67c6381f04db5b84e9ecdacab28bc19f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-1.ec2.internal" Sep 08 08:09:47.498852 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:47.498808 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-1.ec2.internal\" not found" Sep 08 08:09:47.599037 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:47.599008 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-1.ec2.internal\" not found" Sep 08 08:09:47.602162 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.602150 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-1.ec2.internal" Sep 08 08:09:47.606599 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.606573 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-1.ec2.internal" Sep 08 08:09:47.699631 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:47.699606 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-1.ec2.internal\" not found" Sep 08 08:09:47.800149 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:47.800108 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-1.ec2.internal\" not found" Sep 08 08:09:47.825195 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.825178 2585 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 08:09:47.851006 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.850988 2585 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 08:09:47.900755 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:47.900736 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-1.ec2.internal\" not found" Sep 08 08:09:47.956136 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.956115 2585 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 08 08:09:47.956509 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.956221 2585 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 08 08:09:47.956509 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.956265 2585 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 08 08:09:47.956509 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.956266 2585 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 08 08:09:47.998957 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:47.998939 2585 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 08:09:48.015310 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:48.015295 2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-1.ec2.internal" Sep 08 08:09:48.015437 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:48.015324 2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 08 08:09:48.024192 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:48.024171 2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 08 08:09:48.027423 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:48.027393 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-07 08:04:47 +0000 UTC" deadline="2028-02-24 23:01:06.167890744 +0000 UTC" Sep 08 08:09:48.027423 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:48.027421 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12830h51m18.140474342s" Sep 08 08:09:48.027590 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:48.027439 2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 08 08:09:48.028520 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:48.028503 2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-1.ec2.internal" Sep 08 08:09:48.035135 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:48.035118 2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 08 08:09:48.046180 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:48.046161 2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-jl9fq" Sep 08 08:09:48.054178 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:48.054083 2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-jl9fq" Sep 08 08:09:48.114884 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:48.114843 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67c6381f04db5b84e9ecdacab28bc19f.slice/crio-999ea2b2303009883293cb164e74161cf49e5245fa9fffe0865abcb846286d9d WatchSource:0}: Error finding container 999ea2b2303009883293cb164e74161cf49e5245fa9fffe0865abcb846286d9d: Status 404 returned error can't find the container with id 999ea2b2303009883293cb164e74161cf49e5245fa9fffe0865abcb846286d9d Sep 08 08:09:48.115574 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:48.115554 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f38569e02cf6895d157f6aa28cad740.slice/crio-84df49f54d806689bdb7ff3ee98344b683eb848bf832ea99737bd5b9e5055bbb WatchSource:0}: Error finding container 84df49f54d806689bdb7ff3ee98344b683eb848bf832ea99737bd5b9e5055bbb: Status 404 returned error can't find the container with id 84df49f54d806689bdb7ff3ee98344b683eb848bf832ea99737bd5b9e5055bbb Sep 08 08:09:48.120673 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:48.120656 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 08 08:09:48.172326 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:48.172283 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-1.ec2.internal" event={"ID":"6f38569e02cf6895d157f6aa28cad740","Type":"ContainerStarted","Data":"84df49f54d806689bdb7ff3ee98344b683eb848bf832ea99737bd5b9e5055bbb"} Sep 08 08:09:48.173277 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:48.173259 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-1.ec2.internal" event={"ID":"67c6381f04db5b84e9ecdacab28bc19f","Type":"ContainerStarted","Data":"999ea2b2303009883293cb164e74161cf49e5245fa9fffe0865abcb846286d9d"} Sep 08 08:09:48.997784 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:48.997749 2585 apiserver.go:52] "Watching apiserver" Sep 08 08:09:49.006135 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.006106 2585 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 08 08:09:49.006605 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.006574 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-hnpc6","openshift-dns/node-resolver-bxkxt","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-1.ec2.internal","openshift-network-diagnostics/network-check-target-k45rp","openshift-network-operator/iptables-alerter-hhv6k","openshift-ovn-kubernetes/ovnkube-node-2xlvk","kube-system/konnectivity-agent-ffdd8","kube-system/kube-apiserver-proxy-ip-10-0-143-1.ec2.internal","openshift-image-registry/node-ca-jqfnk","openshift-multus/multus-2r8ll","openshift-multus/multus-additional-cni-plugins-zwkjc","openshift-multus/network-metrics-daemon-ns5tq","kube-system/global-pull-secret-syncer-mczjr","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7snsl"] Sep 08 08:09:49.008338 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.008318 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.009656 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.009635 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bxkxt" Sep 08 08:09:49.010864 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.010840 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7snsl" Sep 08 08:09:49.010971 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.010896 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-qtqmt\"" Sep 08 08:09:49.010971 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.010941 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 08 08:09:49.011262 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.011243 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 08 08:09:49.012592 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.012569 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-f5czd\"" Sep 08 08:09:49.012739 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.012601 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 08 08:09:49.012810 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.012782 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 08 08:09:49.013095 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.013077 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:09:49.013229 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:49.013201 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns5tq" podUID="95940b17-787c-4892-96f0-83cf9501e4d5" Sep 08 08:09:49.013706 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.013685 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-bqjj6\"" Sep 08 08:09:49.013775 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.013726 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 08 08:09:49.014001 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.013983 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 08 08:09:49.014197 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.014178 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 08 08:09:49.014523 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.014489 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ffdd8" Sep 08 08:09:49.015857 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.015836 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.017470 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.017449 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hn4d7\"" Sep 08 08:09:49.017702 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.017688 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 08 08:09:49.017978 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.017963 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 08 08:09:49.021692 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.021674 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hhv6k" Sep 08 08:09:49.022716 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.022694 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 08 08:09:49.022996 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.022971 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 08 08:09:49.023259 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.023239 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 08 08:09:49.023572 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.023541 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-sq8mr\"" Sep 08 08:09:49.023946 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.023929 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 08 08:09:49.024268 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.024248 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zwkjc" Sep 08 08:09:49.024348 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.024270 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 08 08:09:49.024402 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.024365 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 08 08:09:49.024402 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.024391 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 08 08:09:49.024602 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.024582 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-x4nzq\"" Sep 08 08:09:49.024741 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.024715 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 08 08:09:49.024817 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.024760 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 08 08:09:49.024817 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.024784 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jqfnk" Sep 08 08:09:49.025089 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.025071 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.025554 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.025533 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:09:49.025841 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:49.025756 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k45rp" podUID="7301238b-b7d9-4101-8f7a-5e442499ca05" Sep 08 08:09:49.027836 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.027708 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:09:49.027836 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.027737 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 08 08:09:49.027836 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:49.027767 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mczjr" podUID="dac5eaa3-6c21-4b05-ace5-a13c8344ab6e" Sep 08 08:09:49.028079 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.027897 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 08 08:09:49.028079 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.027971 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 08 08:09:49.028079 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.028030 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 08 08:09:49.028271 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.028251 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 08 08:09:49.028531 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.028456 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pqtnn\"" Sep 08 08:09:49.028531 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.028484 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 08 08:09:49.028702 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.028682 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 08 08:09:49.029181 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.028796 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 08 08:09:49.029181 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.028816 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-xkp8w\"" Sep 08 08:09:49.029181 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.029076 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 08 08:09:49.030886 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.029358 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-7ww7j\"" Sep 08 08:09:49.033622 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.033602 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-host-kubelet\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.033727 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.033637 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-var-lib-openvswitch\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.033727 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.033664 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-host-cni-bin\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.033727 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.033689 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7zxb\" (UniqueName: \"kubernetes.io/projected/e00cae51-f771-4039-81ec-54fa2a48e98f-kube-api-access-x7zxb\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.033889 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.033747 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dc7f3943-6396-455a-8432-5129bb96c701-device-dir\") pod \"aws-ebs-csi-driver-node-7snsl\" (UID: \"dc7f3943-6396-455a-8432-5129bb96c701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7snsl" Sep 08 08:09:49.033889 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.033782 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzr4m\" (UniqueName: \"kubernetes.io/projected/95940b17-787c-4892-96f0-83cf9501e4d5-kube-api-access-qzr4m\") pod \"network-metrics-daemon-ns5tq\" (UID: \"95940b17-787c-4892-96f0-83cf9501e4d5\") " pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:09:49.033889 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.033799 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fbc11a97-76be-4f9f-a3d1-354de4d4f1b5-agent-certs\") pod \"konnectivity-agent-ffdd8\" (UID: \"fbc11a97-76be-4f9f-a3d1-354de4d4f1b5\") " pod="kube-system/konnectivity-agent-ffdd8" Sep 08 08:09:49.033889 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.033815 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f4583a67-e97f-467b-a658-fe2d36c197e2-host-slash\") pod \"iptables-alerter-hhv6k\" (UID: \"f4583a67-e97f-467b-a658-fe2d36c197e2\") " pod="openshift-network-operator/iptables-alerter-hhv6k" Sep 08 08:09:49.033889 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.033840 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9852555a-bf3c-4370-b542-d139f523ebf0-etc-sysconfig\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.033889 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.033865 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9852555a-bf3c-4370-b542-d139f523ebf0-etc-sysctl-conf\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.033889 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.033885 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9852555a-bf3c-4370-b542-d139f523ebf0-run\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.034205 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.033909 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-etc-openvswitch\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.034205 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.033985 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-node-log\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.034205 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.034017 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-host-cni-netd\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.034205 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.034045 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9852555a-bf3c-4370-b542-d139f523ebf0-etc-modprobe-d\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.034205 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.034069 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9852555a-bf3c-4370-b542-d139f523ebf0-etc-sysctl-d\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.034205 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.034093 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9852555a-bf3c-4370-b542-d139f523ebf0-lib-modules\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.034205 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.034133 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvk29\" (UniqueName: \"kubernetes.io/projected/9852555a-bf3c-4370-b542-d139f523ebf0-kube-api-access-bvk29\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.034205 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.034163 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.034205 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.034188 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e00cae51-f771-4039-81ec-54fa2a48e98f-env-overrides\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.034636 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.034210 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9852555a-bf3c-4370-b542-d139f523ebf0-etc-kubernetes\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.034636 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.034234 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9852555a-bf3c-4370-b542-d139f523ebf0-etc-tuned\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.034636 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.034258 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6eee659d-776f-4175-9e5a-6574fdd0e06c-tmp-dir\") pod \"node-resolver-bxkxt\" (UID: \"6eee659d-776f-4175-9e5a-6574fdd0e06c\") " pod="openshift-dns/node-resolver-bxkxt" Sep 08 08:09:49.034636 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.034283 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95940b17-787c-4892-96f0-83cf9501e4d5-metrics-certs\") pod \"network-metrics-daemon-ns5tq\" (UID: \"95940b17-787c-4892-96f0-83cf9501e4d5\") " pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:09:49.034636 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.034308 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-run-systemd\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.034636 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.034344 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dc7f3943-6396-455a-8432-5129bb96c701-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7snsl\" (UID: \"dc7f3943-6396-455a-8432-5129bb96c701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7snsl" Sep 08 08:09:49.034636 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.034377 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dc7f3943-6396-455a-8432-5129bb96c701-sys-fs\") pod \"aws-ebs-csi-driver-node-7snsl\" (UID: \"dc7f3943-6396-455a-8432-5129bb96c701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7snsl" Sep 08 08:09:49.034636 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.034419 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd5tn\" (UniqueName: \"kubernetes.io/projected/dc7f3943-6396-455a-8432-5129bb96c701-kube-api-access-qd5tn\") pod \"aws-ebs-csi-driver-node-7snsl\" (UID: \"dc7f3943-6396-455a-8432-5129bb96c701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7snsl" Sep 08 08:09:49.034636 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.034446 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-log-socket\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.034636 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.034470 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-host-run-ovn-kubernetes\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.034636 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.034494 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e00cae51-f771-4039-81ec-54fa2a48e98f-ovn-node-metrics-cert\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.034636 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.034518 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e00cae51-f771-4039-81ec-54fa2a48e98f-ovnkube-script-lib\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.034636 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.034554 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dc7f3943-6396-455a-8432-5129bb96c701-socket-dir\") pod \"aws-ebs-csi-driver-node-7snsl\" (UID: \"dc7f3943-6396-455a-8432-5129bb96c701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7snsl" Sep 08 08:09:49.034636 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.034579 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dc7f3943-6396-455a-8432-5129bb96c701-registration-dir\") pod \"aws-ebs-csi-driver-node-7snsl\" (UID: \"dc7f3943-6396-455a-8432-5129bb96c701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7snsl" Sep 08 08:09:49.034636 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.034613 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fbc11a97-76be-4f9f-a3d1-354de4d4f1b5-konnectivity-ca\") pod \"konnectivity-agent-ffdd8\" (UID: \"fbc11a97-76be-4f9f-a3d1-354de4d4f1b5\") " pod="kube-system/konnectivity-agent-ffdd8" Sep 08 08:09:49.034636 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.034636 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9852555a-bf3c-4370-b542-d139f523ebf0-tmp\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.035185 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.034660 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6eee659d-776f-4175-9e5a-6574fdd0e06c-hosts-file\") pod \"node-resolver-bxkxt\" (UID: \"6eee659d-776f-4175-9e5a-6574fdd0e06c\") " pod="openshift-dns/node-resolver-bxkxt" Sep 08 08:09:49.035185 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.034689 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-systemd-units\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.035185 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.034714 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-host-run-netns\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.035185 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.034755 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-run-openvswitch\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.035185 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.034787 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e00cae51-f771-4039-81ec-54fa2a48e98f-ovnkube-config\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.035185 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.034820 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f4583a67-e97f-467b-a658-fe2d36c197e2-iptables-alerter-script\") pod \"iptables-alerter-hhv6k\" (UID: \"f4583a67-e97f-467b-a658-fe2d36c197e2\") " pod="openshift-network-operator/iptables-alerter-hhv6k" Sep 08 08:09:49.035185 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.034907 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvg8h\" (UniqueName: \"kubernetes.io/projected/f4583a67-e97f-467b-a658-fe2d36c197e2-kube-api-access-dvg8h\") pod \"iptables-alerter-hhv6k\" (UID: \"f4583a67-e97f-467b-a658-fe2d36c197e2\") " pod="openshift-network-operator/iptables-alerter-hhv6k" Sep 08 08:09:49.035185 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.034943 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-host-slash\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.035185 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.034984 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9852555a-bf3c-4370-b542-d139f523ebf0-etc-systemd\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.035185 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.035013 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9852555a-bf3c-4370-b542-d139f523ebf0-host\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.035185 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.035037 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-run-ovn\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.035185 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.035066 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dc7f3943-6396-455a-8432-5129bb96c701-etc-selinux\") pod \"aws-ebs-csi-driver-node-7snsl\" (UID: \"dc7f3943-6396-455a-8432-5129bb96c701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7snsl" Sep 08 08:09:49.035185 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.035088 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9852555a-bf3c-4370-b542-d139f523ebf0-sys\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.035185 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.035112 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9852555a-bf3c-4370-b542-d139f523ebf0-var-lib-kubelet\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.035185 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.035136 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8k78\" (UniqueName: \"kubernetes.io/projected/6eee659d-776f-4175-9e5a-6574fdd0e06c-kube-api-access-n8k78\") pod \"node-resolver-bxkxt\" (UID: \"6eee659d-776f-4175-9e5a-6574fdd0e06c\") " pod="openshift-dns/node-resolver-bxkxt" Sep 08 08:09:49.057567 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.057544 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 08:04:48 +0000 UTC" deadline="2028-02-02 02:11:19.482738076 +0000 UTC" Sep 08 08:09:49.057567 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.057566 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12282h1m30.425174737s" Sep 08 08:09:49.117300 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.117270 2585 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 08 08:09:49.135689 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.135658 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dc7f3943-6396-455a-8432-5129bb96c701-socket-dir\") pod \"aws-ebs-csi-driver-node-7snsl\" (UID: \"dc7f3943-6396-455a-8432-5129bb96c701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7snsl" Sep 08 08:09:49.135823 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.135694 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dc7f3943-6396-455a-8432-5129bb96c701-registration-dir\") pod \"aws-ebs-csi-driver-node-7snsl\" (UID: \"dc7f3943-6396-455a-8432-5129bb96c701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7snsl" Sep 08 08:09:49.135823 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.135724 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-multus-socket-dir-parent\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.135823 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.135751 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-multus-conf-dir\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.135823 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.135774 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9852555a-bf3c-4370-b542-d139f523ebf0-tmp\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.136036 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.135818 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6eee659d-776f-4175-9e5a-6574fdd0e06c-hosts-file\") pod \"node-resolver-bxkxt\" (UID: \"6eee659d-776f-4175-9e5a-6574fdd0e06c\") " pod="openshift-dns/node-resolver-bxkxt" Sep 08 08:09:49.136036 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.135826 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dc7f3943-6396-455a-8432-5129bb96c701-registration-dir\") pod \"aws-ebs-csi-driver-node-7snsl\" (UID: \"dc7f3943-6396-455a-8432-5129bb96c701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7snsl" Sep 08 08:09:49.136036 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.135855 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-run-openvswitch\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.136036 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.135874 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dc7f3943-6396-455a-8432-5129bb96c701-socket-dir\") pod \"aws-ebs-csi-driver-node-7snsl\" (UID: \"dc7f3943-6396-455a-8432-5129bb96c701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7snsl" Sep 08 08:09:49.136036 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.135923 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6eee659d-776f-4175-9e5a-6574fdd0e06c-hosts-file\") pod \"node-resolver-bxkxt\" (UID: \"6eee659d-776f-4175-9e5a-6574fdd0e06c\") " pod="openshift-dns/node-resolver-bxkxt" Sep 08 08:09:49.136036 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.135930 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f4583a67-e97f-467b-a658-fe2d36c197e2-iptables-alerter-script\") pod \"iptables-alerter-hhv6k\" (UID: \"f4583a67-e97f-467b-a658-fe2d36c197e2\") " pod="openshift-network-operator/iptables-alerter-hhv6k" Sep 08 08:09:49.136036 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.135954 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-run-openvswitch\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.136036 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.135991 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvg8h\" (UniqueName: \"kubernetes.io/projected/f4583a67-e97f-467b-a658-fe2d36c197e2-kube-api-access-dvg8h\") pod \"iptables-alerter-hhv6k\" (UID: \"f4583a67-e97f-467b-a658-fe2d36c197e2\") " pod="openshift-network-operator/iptables-alerter-hhv6k" Sep 08 08:09:49.136411 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.136047 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dac5eaa3-6c21-4b05-ace5-a13c8344ab6e-kubelet-config\") pod \"global-pull-secret-syncer-mczjr\" (UID: \"dac5eaa3-6c21-4b05-ace5-a13c8344ab6e\") " pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:09:49.136411 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.136164 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-host-var-lib-kubelet\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.136411 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.136190 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f66a9794-3b59-486b-ba0a-348a31d3b263-multus-daemon-config\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.136411 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.136239 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9852555a-bf3c-4370-b542-d139f523ebf0-host\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.136411 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.136269 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fcd3e62c-7a33-4211-9b1f-1223aeb19d09-system-cni-dir\") pod \"multus-additional-cni-plugins-zwkjc\" (UID: \"fcd3e62c-7a33-4211-9b1f-1223aeb19d09\") " pod="openshift-multus/multus-additional-cni-plugins-zwkjc" Sep 08 08:09:49.136411 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.136324 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9852555a-bf3c-4370-b542-d139f523ebf0-host\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.136411 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.136358 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fcd3e62c-7a33-4211-9b1f-1223aeb19d09-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zwkjc\" (UID: \"fcd3e62c-7a33-4211-9b1f-1223aeb19d09\") " pod="openshift-multus/multus-additional-cni-plugins-zwkjc" Sep 08 08:09:49.136807 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.136416 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9852555a-bf3c-4370-b542-d139f523ebf0-var-lib-kubelet\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.136807 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.136444 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-host-cni-bin\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.136807 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.136470 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-cnibin\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.136807 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.136474 2585 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 08 08:09:49.136807 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.136500 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-host-run-multus-certs\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.136807 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.136526 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-etc-kubernetes\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.136807 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.136541 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9852555a-bf3c-4370-b542-d139f523ebf0-var-lib-kubelet\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.136807 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.136553 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzr4m\" (UniqueName: \"kubernetes.io/projected/95940b17-787c-4892-96f0-83cf9501e4d5-kube-api-access-qzr4m\") pod \"network-metrics-daemon-ns5tq\" (UID: \"95940b17-787c-4892-96f0-83cf9501e4d5\") " pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:09:49.136807 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.136558 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f4583a67-e97f-467b-a658-fe2d36c197e2-iptables-alerter-script\") pod \"iptables-alerter-hhv6k\" (UID: \"f4583a67-e97f-467b-a658-fe2d36c197e2\") " pod="openshift-network-operator/iptables-alerter-hhv6k" Sep 08 08:09:49.136807 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.136577 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9852555a-bf3c-4370-b542-d139f523ebf0-run\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.136807 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.136607 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-etc-openvswitch\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.136807 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.136633 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dac5eaa3-6c21-4b05-ace5-a13c8344ab6e-dbus\") pod \"global-pull-secret-syncer-mczjr\" (UID: \"dac5eaa3-6c21-4b05-ace5-a13c8344ab6e\") " pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:09:49.136807 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.136657 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dac5eaa3-6c21-4b05-ace5-a13c8344ab6e-original-pull-secret\") pod \"global-pull-secret-syncer-mczjr\" (UID: \"dac5eaa3-6c21-4b05-ace5-a13c8344ab6e\") " pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:09:49.136807 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.136683 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdbz5\" (UniqueName: \"kubernetes.io/projected/6698964c-4407-4737-a7ca-c673a3feef82-kube-api-access-rdbz5\") pod \"node-ca-jqfnk\" (UID: \"6698964c-4407-4737-a7ca-c673a3feef82\") " pod="openshift-image-registry/node-ca-jqfnk" Sep 08 08:09:49.136807 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.136717 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-host-run-k8s-cni-cncf-io\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.137533 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.137506 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-host-var-lib-cni-bin\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.137681 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.137658 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9852555a-bf3c-4370-b542-d139f523ebf0-etc-modprobe-d\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.137750 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.136747 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9852555a-bf3c-4370-b542-d139f523ebf0-run\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.137750 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.137706 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9852555a-bf3c-4370-b542-d139f523ebf0-etc-sysctl-d\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.137750 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.137737 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvk29\" (UniqueName: \"kubernetes.io/projected/9852555a-bf3c-4370-b542-d139f523ebf0-kube-api-access-bvk29\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.137844 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.137774 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e00cae51-f771-4039-81ec-54fa2a48e98f-env-overrides\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.137844 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.137776 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-host-cni-bin\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.137844 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.137801 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9852555a-bf3c-4370-b542-d139f523ebf0-etc-tuned\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.137844 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.137826 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95940b17-787c-4892-96f0-83cf9501e4d5-metrics-certs\") pod \"network-metrics-daemon-ns5tq\" (UID: \"95940b17-787c-4892-96f0-83cf9501e4d5\") " pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:09:49.137974 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.137862 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-run-systemd\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.137974 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.137900 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9852555a-bf3c-4370-b542-d139f523ebf0-etc-sysctl-d\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.138038 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:49.138025 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 08:09:49.138109 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:49.138096 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95940b17-787c-4892-96f0-83cf9501e4d5-metrics-certs podName:95940b17-787c-4892-96f0-83cf9501e4d5 nodeName:}" failed. No retries permitted until 2026-09-08 08:09:49.638076093 +0000 UTC m=+2.975715730 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/95940b17-787c-4892-96f0-83cf9501e4d5-metrics-certs") pod "network-metrics-daemon-ns5tq" (UID: "95940b17-787c-4892-96f0-83cf9501e4d5") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 08:09:49.138183 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.138164 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9852555a-bf3c-4370-b542-d139f523ebf0-etc-modprobe-d\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.138578 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.138483 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-run-systemd\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.138684 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.138668 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e00cae51-f771-4039-81ec-54fa2a48e98f-env-overrides\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.140405 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.140376 2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 08:09:49.140705 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.140686 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-etc-openvswitch\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.140920 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.140902 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qd5tn\" (UniqueName: \"kubernetes.io/projected/dc7f3943-6396-455a-8432-5129bb96c701-kube-api-access-qd5tn\") pod \"aws-ebs-csi-driver-node-7snsl\" (UID: \"dc7f3943-6396-455a-8432-5129bb96c701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7snsl" Sep 08 08:09:49.140959 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.140947 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv4sj\" (UniqueName: \"kubernetes.io/projected/f66a9794-3b59-486b-ba0a-348a31d3b263-kube-api-access-xv4sj\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.140998 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.140980 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fcd3e62c-7a33-4211-9b1f-1223aeb19d09-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zwkjc\" (UID: \"fcd3e62c-7a33-4211-9b1f-1223aeb19d09\") " pod="openshift-multus/multus-additional-cni-plugins-zwkjc" Sep 08 08:09:49.141061 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.141017 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-host-run-ovn-kubernetes\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.141061 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.141051 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fbc11a97-76be-4f9f-a3d1-354de4d4f1b5-konnectivity-ca\") pod \"konnectivity-agent-ffdd8\" (UID: \"fbc11a97-76be-4f9f-a3d1-354de4d4f1b5\") " pod="kube-system/konnectivity-agent-ffdd8" Sep 08 08:09:49.141203 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.141077 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-system-cni-dir\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.141203 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.141097 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-hostroot\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.141203 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.141118 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-systemd-units\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.141203 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.141156 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-host-run-netns\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.141203 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.141189 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e00cae51-f771-4039-81ec-54fa2a48e98f-ovnkube-config\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.141467 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.141212 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-multus-cni-dir\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.141467 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.141211 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9852555a-bf3c-4370-b542-d139f523ebf0-etc-tuned\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.141467 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.141232 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-host-var-lib-cni-multus\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.141467 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.141254 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-host-slash\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.141467 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.141271 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9852555a-bf3c-4370-b542-d139f523ebf0-etc-systemd\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.141467 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.141284 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9852555a-bf3c-4370-b542-d139f523ebf0-tmp\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.141467 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.141297 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-run-ovn\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.141467 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.141377 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-systemd-units\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.141467 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.141405 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-host-slash\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.141467 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.141414 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-run-ovn\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.141467 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.141445 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9852555a-bf3c-4370-b542-d139f523ebf0-etc-systemd\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.142052 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.141527 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-host-run-ovn-kubernetes\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.142052 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.141568 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-host-run-netns\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.142052 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.141609 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dc7f3943-6396-455a-8432-5129bb96c701-etc-selinux\") pod \"aws-ebs-csi-driver-node-7snsl\" (UID: \"dc7f3943-6396-455a-8432-5129bb96c701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7snsl" Sep 08 08:09:49.142052 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.141643 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6698964c-4407-4737-a7ca-c673a3feef82-host\") pod \"node-ca-jqfnk\" (UID: \"6698964c-4407-4737-a7ca-c673a3feef82\") " pod="openshift-image-registry/node-ca-jqfnk" Sep 08 08:09:49.142052 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.141675 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9852555a-bf3c-4370-b542-d139f523ebf0-sys\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.142052 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.141709 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8k78\" (UniqueName: \"kubernetes.io/projected/6eee659d-776f-4175-9e5a-6574fdd0e06c-kube-api-access-n8k78\") pod \"node-resolver-bxkxt\" (UID: \"6eee659d-776f-4175-9e5a-6574fdd0e06c\") " pod="openshift-dns/node-resolver-bxkxt" Sep 08 08:09:49.142052 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.141735 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-host-kubelet\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.142052 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.141766 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-var-lib-openvswitch\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.142052 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.141800 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7zxb\" (UniqueName: \"kubernetes.io/projected/e00cae51-f771-4039-81ec-54fa2a48e98f-kube-api-access-x7zxb\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.142052 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.141832 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dc7f3943-6396-455a-8432-5129bb96c701-device-dir\") pod \"aws-ebs-csi-driver-node-7snsl\" (UID: \"dc7f3943-6396-455a-8432-5129bb96c701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7snsl" Sep 08 08:09:49.142052 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.141865 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fcd3e62c-7a33-4211-9b1f-1223aeb19d09-cnibin\") pod \"multus-additional-cni-plugins-zwkjc\" (UID: \"fcd3e62c-7a33-4211-9b1f-1223aeb19d09\") " pod="openshift-multus/multus-additional-cni-plugins-zwkjc" Sep 08 08:09:49.142052 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.141892 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fbc11a97-76be-4f9f-a3d1-354de4d4f1b5-agent-certs\") pod \"konnectivity-agent-ffdd8\" (UID: \"fbc11a97-76be-4f9f-a3d1-354de4d4f1b5\") " pod="kube-system/konnectivity-agent-ffdd8" Sep 08 08:09:49.142052 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.141929 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f4583a67-e97f-467b-a658-fe2d36c197e2-host-slash\") pod \"iptables-alerter-hhv6k\" (UID: \"f4583a67-e97f-467b-a658-fe2d36c197e2\") " pod="openshift-network-operator/iptables-alerter-hhv6k" Sep 08 08:09:49.142052 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.141963 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54lbw\" (UniqueName: \"kubernetes.io/projected/fcd3e62c-7a33-4211-9b1f-1223aeb19d09-kube-api-access-54lbw\") pod \"multus-additional-cni-plugins-zwkjc\" (UID: \"fcd3e62c-7a33-4211-9b1f-1223aeb19d09\") " pod="openshift-multus/multus-additional-cni-plugins-zwkjc" Sep 08 08:09:49.142052 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.141995 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9852555a-bf3c-4370-b542-d139f523ebf0-etc-sysconfig\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.142052 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.142001 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fbc11a97-76be-4f9f-a3d1-354de4d4f1b5-konnectivity-ca\") pod \"konnectivity-agent-ffdd8\" (UID: \"fbc11a97-76be-4f9f-a3d1-354de4d4f1b5\") " pod="kube-system/konnectivity-agent-ffdd8" Sep 08 08:09:49.142052 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.142028 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9852555a-bf3c-4370-b542-d139f523ebf0-etc-sysctl-conf\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.142904 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.142060 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-node-log\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.142904 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.142070 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e00cae51-f771-4039-81ec-54fa2a48e98f-ovnkube-config\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.142904 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.142103 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-host-cni-netd\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.142904 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.142157 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9852555a-bf3c-4370-b542-d139f523ebf0-sys\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.142904 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.142200 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f66a9794-3b59-486b-ba0a-348a31d3b263-cni-binary-copy\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.142904 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.142254 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fcd3e62c-7a33-4211-9b1f-1223aeb19d09-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zwkjc\" (UID: \"fcd3e62c-7a33-4211-9b1f-1223aeb19d09\") " pod="openshift-multus/multus-additional-cni-plugins-zwkjc" Sep 08 08:09:49.142904 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.142296 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9852555a-bf3c-4370-b542-d139f523ebf0-lib-modules\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.142904 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.142362 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.142904 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.142395 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-host-run-netns\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.142904 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.142432 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlnps\" (UniqueName: \"kubernetes.io/projected/7301238b-b7d9-4101-8f7a-5e442499ca05-kube-api-access-jlnps\") pod \"network-check-target-k45rp\" (UID: \"7301238b-b7d9-4101-8f7a-5e442499ca05\") " pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:09:49.142904 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.142440 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dc7f3943-6396-455a-8432-5129bb96c701-etc-selinux\") pod \"aws-ebs-csi-driver-node-7snsl\" (UID: \"dc7f3943-6396-455a-8432-5129bb96c701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7snsl" Sep 08 08:09:49.142904 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.142543 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-host-kubelet\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.142904 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.142580 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9852555a-bf3c-4370-b542-d139f523ebf0-lib-modules\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.142904 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.142593 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-var-lib-openvswitch\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.142904 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.142635 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9852555a-bf3c-4370-b542-d139f523ebf0-etc-kubernetes\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.142904 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.142665 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-host-cni-netd\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.142904 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.142679 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6eee659d-776f-4175-9e5a-6574fdd0e06c-tmp-dir\") pod \"node-resolver-bxkxt\" (UID: \"6eee659d-776f-4175-9e5a-6574fdd0e06c\") " pod="openshift-dns/node-resolver-bxkxt" Sep 08 08:09:49.143852 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.142750 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dc7f3943-6396-455a-8432-5129bb96c701-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7snsl\" (UID: \"dc7f3943-6396-455a-8432-5129bb96c701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7snsl" Sep 08 08:09:49.143852 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.142787 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dc7f3943-6396-455a-8432-5129bb96c701-sys-fs\") pod \"aws-ebs-csi-driver-node-7snsl\" (UID: \"dc7f3943-6396-455a-8432-5129bb96c701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7snsl" Sep 08 08:09:49.143852 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.142849 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f4583a67-e97f-467b-a658-fe2d36c197e2-host-slash\") pod \"iptables-alerter-hhv6k\" (UID: \"f4583a67-e97f-467b-a658-fe2d36c197e2\") " pod="openshift-network-operator/iptables-alerter-hhv6k" Sep 08 08:09:49.143852 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.142924 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.143852 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.143011 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dc7f3943-6396-455a-8432-5129bb96c701-sys-fs\") pod \"aws-ebs-csi-driver-node-7snsl\" (UID: \"dc7f3943-6396-455a-8432-5129bb96c701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7snsl" Sep 08 08:09:49.143852 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.143065 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9852555a-bf3c-4370-b542-d139f523ebf0-etc-sysconfig\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.143852 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.143097 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6eee659d-776f-4175-9e5a-6574fdd0e06c-tmp-dir\") pod \"node-resolver-bxkxt\" (UID: \"6eee659d-776f-4175-9e5a-6574fdd0e06c\") " pod="openshift-dns/node-resolver-bxkxt" Sep 08 08:09:49.143852 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.143145 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6698964c-4407-4737-a7ca-c673a3feef82-serviceca\") pod \"node-ca-jqfnk\" (UID: \"6698964c-4407-4737-a7ca-c673a3feef82\") " pod="openshift-image-registry/node-ca-jqfnk" Sep 08 08:09:49.143852 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.143181 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9852555a-bf3c-4370-b542-d139f523ebf0-etc-sysctl-conf\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.143852 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.143185 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fcd3e62c-7a33-4211-9b1f-1223aeb19d09-cni-binary-copy\") pod \"multus-additional-cni-plugins-zwkjc\" (UID: \"fcd3e62c-7a33-4211-9b1f-1223aeb19d09\") " pod="openshift-multus/multus-additional-cni-plugins-zwkjc" Sep 08 08:09:49.143852 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.143223 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-node-log\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.143852 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.143225 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-log-socket\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.143852 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.143284 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e00cae51-f771-4039-81ec-54fa2a48e98f-ovn-node-metrics-cert\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.143852 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.143336 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dc7f3943-6396-455a-8432-5129bb96c701-device-dir\") pod \"aws-ebs-csi-driver-node-7snsl\" (UID: \"dc7f3943-6396-455a-8432-5129bb96c701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7snsl" Sep 08 08:09:49.143852 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.143346 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e00cae51-f771-4039-81ec-54fa2a48e98f-ovnkube-script-lib\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.143852 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.143286 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e00cae51-f771-4039-81ec-54fa2a48e98f-log-socket\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.143852 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.143499 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9852555a-bf3c-4370-b542-d139f523ebf0-etc-kubernetes\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.144751 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.144127 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e00cae51-f771-4039-81ec-54fa2a48e98f-ovnkube-script-lib\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.144751 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.144169 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dc7f3943-6396-455a-8432-5129bb96c701-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7snsl\" (UID: \"dc7f3943-6396-455a-8432-5129bb96c701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7snsl" Sep 08 08:09:49.148206 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.148181 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e00cae51-f771-4039-81ec-54fa2a48e98f-ovn-node-metrics-cert\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.150323 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.150294 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fbc11a97-76be-4f9f-a3d1-354de4d4f1b5-agent-certs\") pod \"konnectivity-agent-ffdd8\" (UID: \"fbc11a97-76be-4f9f-a3d1-354de4d4f1b5\") " pod="kube-system/konnectivity-agent-ffdd8" Sep 08 08:09:49.157732 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.157668 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzr4m\" (UniqueName: \"kubernetes.io/projected/95940b17-787c-4892-96f0-83cf9501e4d5-kube-api-access-qzr4m\") pod \"network-metrics-daemon-ns5tq\" (UID: \"95940b17-787c-4892-96f0-83cf9501e4d5\") " pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:09:49.158662 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.158272 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvg8h\" (UniqueName: \"kubernetes.io/projected/f4583a67-e97f-467b-a658-fe2d36c197e2-kube-api-access-dvg8h\") pod \"iptables-alerter-hhv6k\" (UID: \"f4583a67-e97f-467b-a658-fe2d36c197e2\") " pod="openshift-network-operator/iptables-alerter-hhv6k" Sep 08 08:09:49.160874 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.160831 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvk29\" (UniqueName: \"kubernetes.io/projected/9852555a-bf3c-4370-b542-d139f523ebf0-kube-api-access-bvk29\") pod \"tuned-hnpc6\" (UID: \"9852555a-bf3c-4370-b542-d139f523ebf0\") " pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.161207 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.161177 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7zxb\" (UniqueName: \"kubernetes.io/projected/e00cae51-f771-4039-81ec-54fa2a48e98f-kube-api-access-x7zxb\") pod \"ovnkube-node-2xlvk\" (UID: \"e00cae51-f771-4039-81ec-54fa2a48e98f\") " pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.161393 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.161373 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd5tn\" (UniqueName: \"kubernetes.io/projected/dc7f3943-6396-455a-8432-5129bb96c701-kube-api-access-qd5tn\") pod \"aws-ebs-csi-driver-node-7snsl\" (UID: \"dc7f3943-6396-455a-8432-5129bb96c701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7snsl" Sep 08 08:09:49.161792 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.161771 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8k78\" (UniqueName: \"kubernetes.io/projected/6eee659d-776f-4175-9e5a-6574fdd0e06c-kube-api-access-n8k78\") pod \"node-resolver-bxkxt\" (UID: \"6eee659d-776f-4175-9e5a-6574fdd0e06c\") " pod="openshift-dns/node-resolver-bxkxt" Sep 08 08:09:49.244635 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.244605 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dac5eaa3-6c21-4b05-ace5-a13c8344ab6e-kubelet-config\") pod \"global-pull-secret-syncer-mczjr\" (UID: \"dac5eaa3-6c21-4b05-ace5-a13c8344ab6e\") " pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:09:49.244769 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.244647 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-host-var-lib-kubelet\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.244769 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.244675 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f66a9794-3b59-486b-ba0a-348a31d3b263-multus-daemon-config\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.244769 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.244704 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fcd3e62c-7a33-4211-9b1f-1223aeb19d09-system-cni-dir\") pod \"multus-additional-cni-plugins-zwkjc\" (UID: \"fcd3e62c-7a33-4211-9b1f-1223aeb19d09\") " pod="openshift-multus/multus-additional-cni-plugins-zwkjc" Sep 08 08:09:49.244769 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.244731 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fcd3e62c-7a33-4211-9b1f-1223aeb19d09-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zwkjc\" (UID: \"fcd3e62c-7a33-4211-9b1f-1223aeb19d09\") " pod="openshift-multus/multus-additional-cni-plugins-zwkjc" Sep 08 08:09:49.244769 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.244740 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-host-var-lib-kubelet\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.244769 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.244757 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-cnibin\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.244769 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.244740 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dac5eaa3-6c21-4b05-ace5-a13c8344ab6e-kubelet-config\") pod \"global-pull-secret-syncer-mczjr\" (UID: \"dac5eaa3-6c21-4b05-ace5-a13c8344ab6e\") " pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:09:49.245151 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.244784 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fcd3e62c-7a33-4211-9b1f-1223aeb19d09-system-cni-dir\") pod \"multus-additional-cni-plugins-zwkjc\" (UID: \"fcd3e62c-7a33-4211-9b1f-1223aeb19d09\") " pod="openshift-multus/multus-additional-cni-plugins-zwkjc" Sep 08 08:09:49.245151 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.244833 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-host-run-multus-certs\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.245151 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.244783 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-host-run-multus-certs\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.245151 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.244856 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-cnibin\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.245151 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.244874 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-etc-kubernetes\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.245151 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.244901 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dac5eaa3-6c21-4b05-ace5-a13c8344ab6e-dbus\") pod \"global-pull-secret-syncer-mczjr\" (UID: \"dac5eaa3-6c21-4b05-ace5-a13c8344ab6e\") " pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:09:49.245151 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.244931 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dac5eaa3-6c21-4b05-ace5-a13c8344ab6e-original-pull-secret\") pod \"global-pull-secret-syncer-mczjr\" (UID: \"dac5eaa3-6c21-4b05-ace5-a13c8344ab6e\") " pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:09:49.245151 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.244936 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fcd3e62c-7a33-4211-9b1f-1223aeb19d09-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zwkjc\" (UID: \"fcd3e62c-7a33-4211-9b1f-1223aeb19d09\") " pod="openshift-multus/multus-additional-cni-plugins-zwkjc" Sep 08 08:09:49.245151 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.244956 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdbz5\" (UniqueName: \"kubernetes.io/projected/6698964c-4407-4737-a7ca-c673a3feef82-kube-api-access-rdbz5\") pod \"node-ca-jqfnk\" (UID: \"6698964c-4407-4737-a7ca-c673a3feef82\") " pod="openshift-image-registry/node-ca-jqfnk" Sep 08 08:09:49.245151 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.244982 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-host-run-k8s-cni-cncf-io\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.245151 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.244992 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-etc-kubernetes\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.245151 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.245005 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-host-var-lib-cni-bin\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.245151 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.245059 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xv4sj\" (UniqueName: \"kubernetes.io/projected/f66a9794-3b59-486b-ba0a-348a31d3b263-kube-api-access-xv4sj\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.245151 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.245081 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dac5eaa3-6c21-4b05-ace5-a13c8344ab6e-dbus\") pod \"global-pull-secret-syncer-mczjr\" (UID: \"dac5eaa3-6c21-4b05-ace5-a13c8344ab6e\") " pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:09:49.245151 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:49.245100 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 08:09:49.245151 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.245086 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fcd3e62c-7a33-4211-9b1f-1223aeb19d09-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zwkjc\" (UID: \"fcd3e62c-7a33-4211-9b1f-1223aeb19d09\") " pod="openshift-multus/multus-additional-cni-plugins-zwkjc" Sep 08 08:09:49.245151 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.245137 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-system-cni-dir\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.246108 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:49.245169 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dac5eaa3-6c21-4b05-ace5-a13c8344ab6e-original-pull-secret podName:dac5eaa3-6c21-4b05-ace5-a13c8344ab6e nodeName:}" failed. No retries permitted until 2026-09-08 08:09:49.745148482 +0000 UTC m=+3.082788116 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dac5eaa3-6c21-4b05-ace5-a13c8344ab6e-original-pull-secret") pod "global-pull-secret-syncer-mczjr" (UID: "dac5eaa3-6c21-4b05-ace5-a13c8344ab6e") : object "kube-system"/"original-pull-secret" not registered Sep 08 08:09:49.246108 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.245187 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-system-cni-dir\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.246108 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.245187 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-hostroot\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.246108 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.245213 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-multus-cni-dir\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.246108 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.245223 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-hostroot\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.246108 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.245232 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-host-var-lib-cni-multus\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.246108 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.245270 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6698964c-4407-4737-a7ca-c673a3feef82-host\") pod \"node-ca-jqfnk\" (UID: \"6698964c-4407-4737-a7ca-c673a3feef82\") " pod="openshift-image-registry/node-ca-jqfnk" Sep 08 08:09:49.246108 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.245277 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-host-var-lib-cni-multus\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.246108 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.245283 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-multus-cni-dir\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.246108 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.245305 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fcd3e62c-7a33-4211-9b1f-1223aeb19d09-cnibin\") pod \"multus-additional-cni-plugins-zwkjc\" (UID: \"fcd3e62c-7a33-4211-9b1f-1223aeb19d09\") " pod="openshift-multus/multus-additional-cni-plugins-zwkjc" Sep 08 08:09:49.246108 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.245313 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-host-run-k8s-cni-cncf-io\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.246108 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.245337 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-host-var-lib-cni-bin\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.246108 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.245352 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fcd3e62c-7a33-4211-9b1f-1223aeb19d09-cnibin\") pod \"multus-additional-cni-plugins-zwkjc\" (UID: \"fcd3e62c-7a33-4211-9b1f-1223aeb19d09\") " pod="openshift-multus/multus-additional-cni-plugins-zwkjc" Sep 08 08:09:49.246108 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.245355 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54lbw\" (UniqueName: \"kubernetes.io/projected/fcd3e62c-7a33-4211-9b1f-1223aeb19d09-kube-api-access-54lbw\") pod \"multus-additional-cni-plugins-zwkjc\" (UID: \"fcd3e62c-7a33-4211-9b1f-1223aeb19d09\") " pod="openshift-multus/multus-additional-cni-plugins-zwkjc" Sep 08 08:09:49.246108 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.245392 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6698964c-4407-4737-a7ca-c673a3feef82-host\") pod \"node-ca-jqfnk\" (UID: \"6698964c-4407-4737-a7ca-c673a3feef82\") " pod="openshift-image-registry/node-ca-jqfnk" Sep 08 08:09:49.246108 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.245395 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f66a9794-3b59-486b-ba0a-348a31d3b263-cni-binary-copy\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.246108 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.245442 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fcd3e62c-7a33-4211-9b1f-1223aeb19d09-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zwkjc\" (UID: \"fcd3e62c-7a33-4211-9b1f-1223aeb19d09\") " pod="openshift-multus/multus-additional-cni-plugins-zwkjc" Sep 08 08:09:49.246812 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.245467 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-host-run-netns\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.246812 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.245495 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlnps\" (UniqueName: \"kubernetes.io/projected/7301238b-b7d9-4101-8f7a-5e442499ca05-kube-api-access-jlnps\") pod \"network-check-target-k45rp\" (UID: \"7301238b-b7d9-4101-8f7a-5e442499ca05\") " pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:09:49.246812 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.245524 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6698964c-4407-4737-a7ca-c673a3feef82-serviceca\") pod \"node-ca-jqfnk\" (UID: \"6698964c-4407-4737-a7ca-c673a3feef82\") " pod="openshift-image-registry/node-ca-jqfnk" Sep 08 08:09:49.246812 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.245540 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fcd3e62c-7a33-4211-9b1f-1223aeb19d09-cni-binary-copy\") pod \"multus-additional-cni-plugins-zwkjc\" (UID: \"fcd3e62c-7a33-4211-9b1f-1223aeb19d09\") " pod="openshift-multus/multus-additional-cni-plugins-zwkjc" Sep 08 08:09:49.246812 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.245571 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-multus-socket-dir-parent\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.246812 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.245587 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-multus-conf-dir\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.246812 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.245651 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-multus-conf-dir\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.246812 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.245669 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fcd3e62c-7a33-4211-9b1f-1223aeb19d09-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zwkjc\" (UID: \"fcd3e62c-7a33-4211-9b1f-1223aeb19d09\") " pod="openshift-multus/multus-additional-cni-plugins-zwkjc" Sep 08 08:09:49.246812 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.245848 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-host-run-netns\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.246812 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.245918 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f66a9794-3b59-486b-ba0a-348a31d3b263-multus-socket-dir-parent\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.246812 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.246081 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6698964c-4407-4737-a7ca-c673a3feef82-serviceca\") pod \"node-ca-jqfnk\" (UID: \"6698964c-4407-4737-a7ca-c673a3feef82\") " pod="openshift-image-registry/node-ca-jqfnk" Sep 08 08:09:49.246812 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.246137 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fcd3e62c-7a33-4211-9b1f-1223aeb19d09-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zwkjc\" (UID: \"fcd3e62c-7a33-4211-9b1f-1223aeb19d09\") " pod="openshift-multus/multus-additional-cni-plugins-zwkjc" Sep 08 08:09:49.246812 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.246166 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fcd3e62c-7a33-4211-9b1f-1223aeb19d09-cni-binary-copy\") pod \"multus-additional-cni-plugins-zwkjc\" (UID: \"fcd3e62c-7a33-4211-9b1f-1223aeb19d09\") " pod="openshift-multus/multus-additional-cni-plugins-zwkjc" Sep 08 08:09:49.246812 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.246442 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f66a9794-3b59-486b-ba0a-348a31d3b263-multus-daemon-config\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.247284 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.246840 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f66a9794-3b59-486b-ba0a-348a31d3b263-cni-binary-copy\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.253192 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:49.253135 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 08:09:49.253192 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:49.253167 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 08:09:49.253192 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:49.253183 2585 projected.go:196] Error preparing data for projected volume kube-api-access-jlnps for pod openshift-network-diagnostics/network-check-target-k45rp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 08:09:49.253450 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:49.253263 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7301238b-b7d9-4101-8f7a-5e442499ca05-kube-api-access-jlnps podName:7301238b-b7d9-4101-8f7a-5e442499ca05 nodeName:}" failed. No retries permitted until 2026-09-08 08:09:49.753245026 +0000 UTC m=+3.090884659 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-jlnps" (UniqueName: "kubernetes.io/projected/7301238b-b7d9-4101-8f7a-5e442499ca05-kube-api-access-jlnps") pod "network-check-target-k45rp" (UID: "7301238b-b7d9-4101-8f7a-5e442499ca05") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 08:09:49.255174 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.255152 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdbz5\" (UniqueName: \"kubernetes.io/projected/6698964c-4407-4737-a7ca-c673a3feef82-kube-api-access-rdbz5\") pod \"node-ca-jqfnk\" (UID: \"6698964c-4407-4737-a7ca-c673a3feef82\") " pod="openshift-image-registry/node-ca-jqfnk" Sep 08 08:09:49.255753 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.255732 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54lbw\" (UniqueName: \"kubernetes.io/projected/fcd3e62c-7a33-4211-9b1f-1223aeb19d09-kube-api-access-54lbw\") pod \"multus-additional-cni-plugins-zwkjc\" (UID: \"fcd3e62c-7a33-4211-9b1f-1223aeb19d09\") " pod="openshift-multus/multus-additional-cni-plugins-zwkjc" Sep 08 08:09:49.255877 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.255855 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv4sj\" (UniqueName: \"kubernetes.io/projected/f66a9794-3b59-486b-ba0a-348a31d3b263-kube-api-access-xv4sj\") pod \"multus-2r8ll\" (UID: \"f66a9794-3b59-486b-ba0a-348a31d3b263\") " pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.329497 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.329470 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" Sep 08 08:09:49.341320 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.341300 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bxkxt" Sep 08 08:09:49.350552 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.350529 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7snsl" Sep 08 08:09:49.356254 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.356234 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ffdd8" Sep 08 08:09:49.365077 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.365048 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:09:49.373728 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.373706 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hhv6k" Sep 08 08:09:49.381387 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.381367 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zwkjc" Sep 08 08:09:49.390209 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.390187 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jqfnk" Sep 08 08:09:49.396038 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.396018 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2r8ll" Sep 08 08:09:49.648930 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.648902 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95940b17-787c-4892-96f0-83cf9501e4d5-metrics-certs\") pod \"network-metrics-daemon-ns5tq\" (UID: \"95940b17-787c-4892-96f0-83cf9501e4d5\") " pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:09:49.649105 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:49.649086 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 08:09:49.649182 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:49.649165 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95940b17-787c-4892-96f0-83cf9501e4d5-metrics-certs podName:95940b17-787c-4892-96f0-83cf9501e4d5 nodeName:}" failed. No retries permitted until 2026-09-08 08:09:50.649144008 +0000 UTC m=+3.986783646 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/95940b17-787c-4892-96f0-83cf9501e4d5-metrics-certs") pod "network-metrics-daemon-ns5tq" (UID: "95940b17-787c-4892-96f0-83cf9501e4d5") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 08:09:49.749795 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.749765 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dac5eaa3-6c21-4b05-ace5-a13c8344ab6e-original-pull-secret\") pod \"global-pull-secret-syncer-mczjr\" (UID: \"dac5eaa3-6c21-4b05-ace5-a13c8344ab6e\") " pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:09:49.749968 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:49.749907 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 08:09:49.750042 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:49.749971 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dac5eaa3-6c21-4b05-ace5-a13c8344ab6e-original-pull-secret podName:dac5eaa3-6c21-4b05-ace5-a13c8344ab6e nodeName:}" failed. No retries permitted until 2026-09-08 08:09:50.749956968 +0000 UTC m=+4.087596594 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dac5eaa3-6c21-4b05-ace5-a13c8344ab6e-original-pull-secret") pod "global-pull-secret-syncer-mczjr" (UID: "dac5eaa3-6c21-4b05-ace5-a13c8344ab6e") : object "kube-system"/"original-pull-secret" not registered Sep 08 08:09:49.850678 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:49.850648 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlnps\" (UniqueName: \"kubernetes.io/projected/7301238b-b7d9-4101-8f7a-5e442499ca05-kube-api-access-jlnps\") pod \"network-check-target-k45rp\" (UID: \"7301238b-b7d9-4101-8f7a-5e442499ca05\") " pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:09:49.850855 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:49.850841 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 08:09:49.850911 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:49.850866 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 08:09:49.850911 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:49.850882 2585 projected.go:196] Error preparing data for projected volume kube-api-access-jlnps for pod openshift-network-diagnostics/network-check-target-k45rp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 08:09:49.851015 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:49.850944 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7301238b-b7d9-4101-8f7a-5e442499ca05-kube-api-access-jlnps podName:7301238b-b7d9-4101-8f7a-5e442499ca05 nodeName:}" failed. No retries permitted until 2026-09-08 08:09:50.850926642 +0000 UTC m=+4.188566293 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-jlnps" (UniqueName: "kubernetes.io/projected/7301238b-b7d9-4101-8f7a-5e442499ca05-kube-api-access-jlnps") pod "network-check-target-k45rp" (UID: "7301238b-b7d9-4101-8f7a-5e442499ca05") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 08:09:50.058223 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:50.058111 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 08:04:48 +0000 UTC" deadline="2028-03-24 23:54:59.456134138 +0000 UTC" Sep 08 08:09:50.058223 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:50.058176 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13527h45m9.397961685s" Sep 08 08:09:50.247893 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:50.247859 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6eee659d_776f_4175_9e5a_6574fdd0e06c.slice/crio-a32a294cc46f353f80c03033df3fb14bae05ce95cc1ea3fb6e632f20ef9b42aa WatchSource:0}: Error finding container a32a294cc46f353f80c03033df3fb14bae05ce95cc1ea3fb6e632f20ef9b42aa: Status 404 returned error can't find the container with id a32a294cc46f353f80c03033df3fb14bae05ce95cc1ea3fb6e632f20ef9b42aa Sep 08 08:09:50.250869 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:50.250757 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4583a67_e97f_467b_a658_fe2d36c197e2.slice/crio-7458a62bc486ed6ba1b68275ea6ddd7e446b8b27d7e2a01c050b58ffce1c7d5d WatchSource:0}: Error finding container 7458a62bc486ed6ba1b68275ea6ddd7e446b8b27d7e2a01c050b58ffce1c7d5d: Status 404 returned error can't find the container with id 7458a62bc486ed6ba1b68275ea6ddd7e446b8b27d7e2a01c050b58ffce1c7d5d Sep 08 08:09:50.253022 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:50.252992 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf66a9794_3b59_486b_ba0a_348a31d3b263.slice/crio-2fb70d1d1a960d139616e20b3d91a2742f94700a5cbab7bd5132c401d74ace83 WatchSource:0}: Error finding container 2fb70d1d1a960d139616e20b3d91a2742f94700a5cbab7bd5132c401d74ace83: Status 404 returned error can't find the container with id 2fb70d1d1a960d139616e20b3d91a2742f94700a5cbab7bd5132c401d74ace83 Sep 08 08:09:50.253242 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:50.253220 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcd3e62c_7a33_4211_9b1f_1223aeb19d09.slice/crio-cd1e7bd841512647959ccf5b978007183b2d2e6e551048cf76ed8397e787a9b5 WatchSource:0}: Error finding container cd1e7bd841512647959ccf5b978007183b2d2e6e551048cf76ed8397e787a9b5: Status 404 returned error can't find the container with id cd1e7bd841512647959ccf5b978007183b2d2e6e551048cf76ed8397e787a9b5 Sep 08 08:09:50.256662 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:09:50.256644 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9852555a_bf3c_4370_b542_d139f523ebf0.slice/crio-c38736b4d7862872314479d43870ed78b4922fb58bced4c63512931268b1dddf WatchSource:0}: Error finding container c38736b4d7862872314479d43870ed78b4922fb58bced4c63512931268b1dddf: Status 404 returned error can't find the container with id c38736b4d7862872314479d43870ed78b4922fb58bced4c63512931268b1dddf Sep 08 08:09:50.354595 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:50.354573 2585 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 08:09:50.658411 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:50.658375 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95940b17-787c-4892-96f0-83cf9501e4d5-metrics-certs\") pod \"network-metrics-daemon-ns5tq\" (UID: \"95940b17-787c-4892-96f0-83cf9501e4d5\") " pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:09:50.658561 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:50.658537 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 08:09:50.658635 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:50.658618 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95940b17-787c-4892-96f0-83cf9501e4d5-metrics-certs podName:95940b17-787c-4892-96f0-83cf9501e4d5 nodeName:}" failed. No retries permitted until 2026-09-08 08:09:52.65859683 +0000 UTC m=+5.996236458 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/95940b17-787c-4892-96f0-83cf9501e4d5-metrics-certs") pod "network-metrics-daemon-ns5tq" (UID: "95940b17-787c-4892-96f0-83cf9501e4d5") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 08:09:50.759207 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:50.759164 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dac5eaa3-6c21-4b05-ace5-a13c8344ab6e-original-pull-secret\") pod \"global-pull-secret-syncer-mczjr\" (UID: \"dac5eaa3-6c21-4b05-ace5-a13c8344ab6e\") " pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:09:50.759351 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:50.759311 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 08:09:50.759440 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:50.759372 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dac5eaa3-6c21-4b05-ace5-a13c8344ab6e-original-pull-secret podName:dac5eaa3-6c21-4b05-ace5-a13c8344ab6e nodeName:}" failed. No retries permitted until 2026-09-08 08:09:52.759353154 +0000 UTC m=+6.096992775 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dac5eaa3-6c21-4b05-ace5-a13c8344ab6e-original-pull-secret") pod "global-pull-secret-syncer-mczjr" (UID: "dac5eaa3-6c21-4b05-ace5-a13c8344ab6e") : object "kube-system"/"original-pull-secret" not registered Sep 08 08:09:50.861352 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:50.860742 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlnps\" (UniqueName: \"kubernetes.io/projected/7301238b-b7d9-4101-8f7a-5e442499ca05-kube-api-access-jlnps\") pod \"network-check-target-k45rp\" (UID: \"7301238b-b7d9-4101-8f7a-5e442499ca05\") " pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:09:50.861352 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:50.860912 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 08:09:50.861352 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:50.860932 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 08:09:50.861352 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:50.860945 2585 projected.go:196] Error preparing data for projected volume kube-api-access-jlnps for pod openshift-network-diagnostics/network-check-target-k45rp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 08:09:50.861352 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:50.861002 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7301238b-b7d9-4101-8f7a-5e442499ca05-kube-api-access-jlnps podName:7301238b-b7d9-4101-8f7a-5e442499ca05 nodeName:}" failed. No retries permitted until 2026-09-08 08:09:52.860983384 +0000 UTC m=+6.198623012 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-jlnps" (UniqueName: "kubernetes.io/projected/7301238b-b7d9-4101-8f7a-5e442499ca05-kube-api-access-jlnps") pod "network-check-target-k45rp" (UID: "7301238b-b7d9-4101-8f7a-5e442499ca05") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 08:09:51.026101 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:51.025482 2585 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 08:09:51.171794 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:51.171767 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:09:51.172591 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:51.171901 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns5tq" podUID="95940b17-787c-4892-96f0-83cf9501e4d5" Sep 08 08:09:51.172591 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:51.172294 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:09:51.172591 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:51.172382 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k45rp" podUID="7301238b-b7d9-4101-8f7a-5e442499ca05" Sep 08 08:09:51.172936 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:51.172778 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:09:51.172936 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:51.172897 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mczjr" podUID="dac5eaa3-6c21-4b05-ace5-a13c8344ab6e" Sep 08 08:09:51.187880 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:51.187206 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-1.ec2.internal" event={"ID":"6f38569e02cf6895d157f6aa28cad740","Type":"ContainerStarted","Data":"7661eae195d0b7c7803f302c6b1d7c6d1c53579bde79bf66864eec3a2d8df239"} Sep 08 08:09:51.195058 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:51.194432 2585 generic.go:356] "Generic (PLEG): container finished" podID="67c6381f04db5b84e9ecdacab28bc19f" containerID="04ed8f64df9e46c67339f5b49d42372a1ef1864d942ced499571042b29e254f8" exitCode=0 Sep 08 08:09:51.195058 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:51.194522 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-1.ec2.internal" event={"ID":"67c6381f04db5b84e9ecdacab28bc19f","Type":"ContainerDied","Data":"04ed8f64df9e46c67339f5b49d42372a1ef1864d942ced499571042b29e254f8"} Sep 08 08:09:51.200754 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:51.200660 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jqfnk" event={"ID":"6698964c-4407-4737-a7ca-c673a3feef82","Type":"ContainerStarted","Data":"8e7a44f1a2fb75c735cf29c9aee09025bb8bd17e0cf34b3764e65206449f9bd2"} Sep 08 08:09:51.209853 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:51.209820 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" event={"ID":"e00cae51-f771-4039-81ec-54fa2a48e98f","Type":"ContainerStarted","Data":"c7cd2d2edc07395b5a1e7d5b7ac10fb5c58c5d807a509e1f05f510e80942a893"} Sep 08 08:09:51.217225 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:51.217166 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-1.ec2.internal" podStartSLOduration=3.217148405 podStartE2EDuration="3.217148405s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 08:09:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:09:51.202269723 +0000 UTC m=+4.539909371" watchObservedRunningTime="2026-09-08 08:09:51.217148405 +0000 UTC m=+4.554788050" Sep 08 08:09:51.220780 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:51.220744 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ffdd8" event={"ID":"fbc11a97-76be-4f9f-a3d1-354de4d4f1b5","Type":"ContainerStarted","Data":"e14ad90738cc553c5ec339df1df965412427da6329f3593a9cfd66ee61ccd9f9"} Sep 08 08:09:51.225666 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:51.225638 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2r8ll" event={"ID":"f66a9794-3b59-486b-ba0a-348a31d3b263","Type":"ContainerStarted","Data":"2fb70d1d1a960d139616e20b3d91a2742f94700a5cbab7bd5132c401d74ace83"} Sep 08 08:09:51.232696 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:51.232666 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bxkxt" event={"ID":"6eee659d-776f-4175-9e5a-6574fdd0e06c","Type":"ContainerStarted","Data":"a32a294cc46f353f80c03033df3fb14bae05ce95cc1ea3fb6e632f20ef9b42aa"} Sep 08 08:09:51.238749 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:51.238721 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7snsl" event={"ID":"dc7f3943-6396-455a-8432-5129bb96c701","Type":"ContainerStarted","Data":"46816a8fdb6d0ee744efa03d96819f0978fde9a76813f48f4d0d9010d7d2ba91"} Sep 08 08:09:51.241836 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:51.241803 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwkjc" event={"ID":"fcd3e62c-7a33-4211-9b1f-1223aeb19d09","Type":"ContainerStarted","Data":"cd1e7bd841512647959ccf5b978007183b2d2e6e551048cf76ed8397e787a9b5"} Sep 08 08:09:51.245660 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:51.245635 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" event={"ID":"9852555a-bf3c-4370-b542-d139f523ebf0","Type":"ContainerStarted","Data":"c38736b4d7862872314479d43870ed78b4922fb58bced4c63512931268b1dddf"} Sep 08 08:09:51.249086 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:51.249060 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hhv6k" event={"ID":"f4583a67-e97f-467b-a658-fe2d36c197e2","Type":"ContainerStarted","Data":"7458a62bc486ed6ba1b68275ea6ddd7e446b8b27d7e2a01c050b58ffce1c7d5d"} Sep 08 08:09:52.276058 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:52.275518 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-1.ec2.internal" event={"ID":"67c6381f04db5b84e9ecdacab28bc19f","Type":"ContainerStarted","Data":"85442e9f1cb13732b1637d65d1f28e0bded2b46f9869e794ea5b7a4b461c6361"} Sep 08 08:09:52.679994 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:52.679960 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95940b17-787c-4892-96f0-83cf9501e4d5-metrics-certs\") pod \"network-metrics-daemon-ns5tq\" (UID: \"95940b17-787c-4892-96f0-83cf9501e4d5\") " pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:09:52.680231 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:52.680124 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 08:09:52.680231 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:52.680221 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95940b17-787c-4892-96f0-83cf9501e4d5-metrics-certs podName:95940b17-787c-4892-96f0-83cf9501e4d5 nodeName:}" failed. No retries permitted until 2026-09-08 08:09:56.680200491 +0000 UTC m=+10.017840117 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/95940b17-787c-4892-96f0-83cf9501e4d5-metrics-certs") pod "network-metrics-daemon-ns5tq" (UID: "95940b17-787c-4892-96f0-83cf9501e4d5") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 08:09:52.780309 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:52.780279 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dac5eaa3-6c21-4b05-ace5-a13c8344ab6e-original-pull-secret\") pod \"global-pull-secret-syncer-mczjr\" (UID: \"dac5eaa3-6c21-4b05-ace5-a13c8344ab6e\") " pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:09:52.780587 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:52.780565 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 08:09:52.780670 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:52.780649 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dac5eaa3-6c21-4b05-ace5-a13c8344ab6e-original-pull-secret podName:dac5eaa3-6c21-4b05-ace5-a13c8344ab6e nodeName:}" failed. No retries permitted until 2026-09-08 08:09:56.780618903 +0000 UTC m=+10.118258541 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dac5eaa3-6c21-4b05-ace5-a13c8344ab6e-original-pull-secret") pod "global-pull-secret-syncer-mczjr" (UID: "dac5eaa3-6c21-4b05-ace5-a13c8344ab6e") : object "kube-system"/"original-pull-secret" not registered Sep 08 08:09:52.881596 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:52.881558 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlnps\" (UniqueName: \"kubernetes.io/projected/7301238b-b7d9-4101-8f7a-5e442499ca05-kube-api-access-jlnps\") pod \"network-check-target-k45rp\" (UID: \"7301238b-b7d9-4101-8f7a-5e442499ca05\") " pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:09:52.882199 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:52.881777 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 08:09:52.882199 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:52.881797 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 08:09:52.882199 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:52.881809 2585 projected.go:196] Error preparing data for projected volume kube-api-access-jlnps for pod openshift-network-diagnostics/network-check-target-k45rp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 08:09:52.882199 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:52.881860 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7301238b-b7d9-4101-8f7a-5e442499ca05-kube-api-access-jlnps podName:7301238b-b7d9-4101-8f7a-5e442499ca05 nodeName:}" failed. No retries permitted until 2026-09-08 08:09:56.881846504 +0000 UTC m=+10.219486125 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-jlnps" (UniqueName: "kubernetes.io/projected/7301238b-b7d9-4101-8f7a-5e442499ca05-kube-api-access-jlnps") pod "network-check-target-k45rp" (UID: "7301238b-b7d9-4101-8f7a-5e442499ca05") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 08:09:53.171149 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:53.171115 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:09:53.171303 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:53.171216 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mczjr" podUID="dac5eaa3-6c21-4b05-ace5-a13c8344ab6e" Sep 08 08:09:53.171632 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:53.171615 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:09:53.171726 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:53.171709 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns5tq" podUID="95940b17-787c-4892-96f0-83cf9501e4d5" Sep 08 08:09:53.171791 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:53.171761 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:09:53.171855 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:53.171837 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k45rp" podUID="7301238b-b7d9-4101-8f7a-5e442499ca05" Sep 08 08:09:55.169963 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:55.169293 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:09:55.169963 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:55.169464 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns5tq" podUID="95940b17-787c-4892-96f0-83cf9501e4d5" Sep 08 08:09:55.169963 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:55.169816 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:09:55.169963 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:55.169911 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k45rp" podUID="7301238b-b7d9-4101-8f7a-5e442499ca05" Sep 08 08:09:55.170601 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:55.170061 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:09:55.170601 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:55.170170 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mczjr" podUID="dac5eaa3-6c21-4b05-ace5-a13c8344ab6e" Sep 08 08:09:56.711995 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:56.711954 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95940b17-787c-4892-96f0-83cf9501e4d5-metrics-certs\") pod \"network-metrics-daemon-ns5tq\" (UID: \"95940b17-787c-4892-96f0-83cf9501e4d5\") " pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:09:56.712500 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:56.712086 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 08:09:56.712500 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:56.712138 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95940b17-787c-4892-96f0-83cf9501e4d5-metrics-certs podName:95940b17-787c-4892-96f0-83cf9501e4d5 nodeName:}" failed. No retries permitted until 2026-09-08 08:10:04.712124541 +0000 UTC m=+18.049764160 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/95940b17-787c-4892-96f0-83cf9501e4d5-metrics-certs") pod "network-metrics-daemon-ns5tq" (UID: "95940b17-787c-4892-96f0-83cf9501e4d5") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 08:09:56.812708 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:56.812644 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dac5eaa3-6c21-4b05-ace5-a13c8344ab6e-original-pull-secret\") pod \"global-pull-secret-syncer-mczjr\" (UID: \"dac5eaa3-6c21-4b05-ace5-a13c8344ab6e\") " pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:09:56.812874 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:56.812822 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 08:09:56.812934 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:56.812898 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dac5eaa3-6c21-4b05-ace5-a13c8344ab6e-original-pull-secret podName:dac5eaa3-6c21-4b05-ace5-a13c8344ab6e nodeName:}" failed. No retries permitted until 2026-09-08 08:10:04.812876966 +0000 UTC m=+18.150516596 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dac5eaa3-6c21-4b05-ace5-a13c8344ab6e-original-pull-secret") pod "global-pull-secret-syncer-mczjr" (UID: "dac5eaa3-6c21-4b05-ace5-a13c8344ab6e") : object "kube-system"/"original-pull-secret" not registered Sep 08 08:09:56.913952 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:56.913915 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlnps\" (UniqueName: \"kubernetes.io/projected/7301238b-b7d9-4101-8f7a-5e442499ca05-kube-api-access-jlnps\") pod \"network-check-target-k45rp\" (UID: \"7301238b-b7d9-4101-8f7a-5e442499ca05\") " pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:09:56.914135 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:56.914099 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 08:09:56.914135 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:56.914124 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 08:09:56.914135 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:56.914137 2585 projected.go:196] Error preparing data for projected volume kube-api-access-jlnps for pod openshift-network-diagnostics/network-check-target-k45rp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 08:09:56.914253 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:56.914194 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7301238b-b7d9-4101-8f7a-5e442499ca05-kube-api-access-jlnps podName:7301238b-b7d9-4101-8f7a-5e442499ca05 nodeName:}" failed. No retries permitted until 2026-09-08 08:10:04.914177392 +0000 UTC m=+18.251817020 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-jlnps" (UniqueName: "kubernetes.io/projected/7301238b-b7d9-4101-8f7a-5e442499ca05-kube-api-access-jlnps") pod "network-check-target-k45rp" (UID: "7301238b-b7d9-4101-8f7a-5e442499ca05") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 08:09:57.169706 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:57.169639 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:09:57.169706 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:57.169639 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:09:57.171125 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:57.170668 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:09:57.171125 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:57.170783 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns5tq" podUID="95940b17-787c-4892-96f0-83cf9501e4d5" Sep 08 08:09:57.171125 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:57.170920 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mczjr" podUID="dac5eaa3-6c21-4b05-ace5-a13c8344ab6e" Sep 08 08:09:57.171125 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:57.171006 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k45rp" podUID="7301238b-b7d9-4101-8f7a-5e442499ca05" Sep 08 08:09:59.169595 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:59.169566 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:09:59.169595 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:59.169595 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:09:59.170094 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:09:59.169570 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:09:59.170094 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:59.169665 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k45rp" podUID="7301238b-b7d9-4101-8f7a-5e442499ca05" Sep 08 08:09:59.170094 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:59.169757 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns5tq" podUID="95940b17-787c-4892-96f0-83cf9501e4d5" Sep 08 08:09:59.170094 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:09:59.169842 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mczjr" podUID="dac5eaa3-6c21-4b05-ace5-a13c8344ab6e" Sep 08 08:10:01.169983 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:01.169428 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:10:01.169983 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:01.169462 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:10:01.169983 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:01.169443 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:10:01.169983 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:01.169617 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns5tq" podUID="95940b17-787c-4892-96f0-83cf9501e4d5" Sep 08 08:10:01.169983 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:01.169690 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mczjr" podUID="dac5eaa3-6c21-4b05-ace5-a13c8344ab6e" Sep 08 08:10:01.169983 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:01.169762 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k45rp" podUID="7301238b-b7d9-4101-8f7a-5e442499ca05" Sep 08 08:10:03.169516 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:03.169310 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:10:03.169516 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:03.169311 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:10:03.169516 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:03.169466 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns5tq" podUID="95940b17-787c-4892-96f0-83cf9501e4d5" Sep 08 08:10:03.169516 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:03.169515 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:10:03.170074 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:03.169601 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mczjr" podUID="dac5eaa3-6c21-4b05-ace5-a13c8344ab6e" Sep 08 08:10:03.170074 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:03.169666 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k45rp" podUID="7301238b-b7d9-4101-8f7a-5e442499ca05" Sep 08 08:10:04.288528 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:04.288489 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7snsl" event={"ID":"dc7f3943-6396-455a-8432-5129bb96c701","Type":"ContainerStarted","Data":"9652ebe879f1d8e5ff9c83c8157f89e9640cb7598cf513fd24d2dce4e21e2d22"} Sep 08 08:10:04.290291 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:04.290262 2585 generic.go:356] "Generic (PLEG): container finished" podID="fcd3e62c-7a33-4211-9b1f-1223aeb19d09" containerID="ce0bb03726d46901eee8e053ba85547cd3889160257d50a63c9f56d9d212e587" exitCode=0 Sep 08 08:10:04.290449 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:04.290355 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwkjc" event={"ID":"fcd3e62c-7a33-4211-9b1f-1223aeb19d09","Type":"ContainerDied","Data":"ce0bb03726d46901eee8e053ba85547cd3889160257d50a63c9f56d9d212e587"} Sep 08 08:10:04.292173 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:04.292150 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" event={"ID":"9852555a-bf3c-4370-b542-d139f523ebf0","Type":"ContainerStarted","Data":"6efd6c70916062d1b7973cde47bc85c85fb8b1168849d2081d1a936c25a53ccd"} Sep 08 08:10:04.293849 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:04.293831 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jqfnk" event={"ID":"6698964c-4407-4737-a7ca-c673a3feef82","Type":"ContainerStarted","Data":"976c1ff0c75f94ad5d1aeb689864a33226353d88a501da63d399397ea221fea2"} Sep 08 08:10:04.295539 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:04.295514 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ffdd8" event={"ID":"fbc11a97-76be-4f9f-a3d1-354de4d4f1b5","Type":"ContainerStarted","Data":"c9314392ec09c4c40747b9c0f37f57c0cfcdead1ed08e0f9ab98144196ff1eae"} Sep 08 08:10:04.297943 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:04.297918 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2r8ll" event={"ID":"f66a9794-3b59-486b-ba0a-348a31d3b263","Type":"ContainerStarted","Data":"c4dc10f55c6b529e96cf0c16f4edf574c8dd7bfc1d2dadd33d133e4392db61d7"} Sep 08 08:10:04.300612 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:04.300587 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bxkxt" event={"ID":"6eee659d-776f-4175-9e5a-6574fdd0e06c","Type":"ContainerStarted","Data":"e79e47ebe0445ace4081933e58006a180af8b3be44e3f12ab909965994e2d5b9"} Sep 08 08:10:04.311773 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:04.311733 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-1.ec2.internal" podStartSLOduration=16.311720542 podStartE2EDuration="16.311720542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 08:09:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:09:52.294074387 +0000 UTC m=+5.631714028" watchObservedRunningTime="2026-09-08 08:10:04.311720542 +0000 UTC m=+17.649360182" Sep 08 08:10:04.341076 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:04.341027 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-jqfnk" podStartSLOduration=2.960935458 podStartE2EDuration="16.341010502s" totalImagesPullingTime="13.380075044s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 08:09:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:10:04.326407992 +0000 UTC m=+17.664047624" watchObservedRunningTime="2026-09-08 08:10:04.341010502 +0000 UTC m=+17.678650144" Sep 08 08:10:04.360799 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:04.360747 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ffdd8" podStartSLOduration=3.026290272 podStartE2EDuration="16.360733434s" totalImagesPullingTime="13.334443162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 08:09:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:10:04.341102942 +0000 UTC m=+17.678742583" watchObservedRunningTime="2026-09-08 08:10:04.360733434 +0000 UTC m=+17.698373074" Sep 08 08:10:04.360920 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:04.360883 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-2r8ll" podStartSLOduration=2.769389456 podStartE2EDuration="16.360877021s" totalImagesPullingTime="13.591487565s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 08:09:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:10:04.360521283 +0000 UTC m=+17.698160924" watchObservedRunningTime="2026-09-08 08:10:04.360877021 +0000 UTC m=+17.698516662" Sep 08 08:10:04.376599 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:04.376557 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-hnpc6" podStartSLOduration=3.988797032 podStartE2EDuration="17.376541621s" totalImagesPullingTime="13.387744589s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 08:09:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:10:04.376231676 +0000 UTC m=+17.713871318" watchObservedRunningTime="2026-09-08 08:10:04.376541621 +0000 UTC m=+17.714181262" Sep 08 08:10:04.390180 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:04.390138 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-bxkxt" podStartSLOduration=3.9918201829999997 podStartE2EDuration="17.390121948s" totalImagesPullingTime="13.398301765s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 08:09:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:10:04.390033069 +0000 UTC m=+17.727672721" watchObservedRunningTime="2026-09-08 08:10:04.390121948 +0000 UTC m=+17.727761590" Sep 08 08:10:04.773938 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:04.773726 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95940b17-787c-4892-96f0-83cf9501e4d5-metrics-certs\") pod \"network-metrics-daemon-ns5tq\" (UID: \"95940b17-787c-4892-96f0-83cf9501e4d5\") " pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:10:04.774106 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:04.773865 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 08:10:04.774106 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:04.774037 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95940b17-787c-4892-96f0-83cf9501e4d5-metrics-certs podName:95940b17-787c-4892-96f0-83cf9501e4d5 nodeName:}" failed. No retries permitted until 2026-09-08 08:10:20.774022197 +0000 UTC m=+34.111661816 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/95940b17-787c-4892-96f0-83cf9501e4d5-metrics-certs") pod "network-metrics-daemon-ns5tq" (UID: "95940b17-787c-4892-96f0-83cf9501e4d5") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 08:10:04.875205 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:04.875125 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dac5eaa3-6c21-4b05-ace5-a13c8344ab6e-original-pull-secret\") pod \"global-pull-secret-syncer-mczjr\" (UID: \"dac5eaa3-6c21-4b05-ace5-a13c8344ab6e\") " pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:10:04.875338 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:04.875246 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 08:10:04.875338 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:04.875305 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dac5eaa3-6c21-4b05-ace5-a13c8344ab6e-original-pull-secret podName:dac5eaa3-6c21-4b05-ace5-a13c8344ab6e nodeName:}" failed. No retries permitted until 2026-09-08 08:10:20.875287775 +0000 UTC m=+34.212927394 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dac5eaa3-6c21-4b05-ace5-a13c8344ab6e-original-pull-secret") pod "global-pull-secret-syncer-mczjr" (UID: "dac5eaa3-6c21-4b05-ace5-a13c8344ab6e") : object "kube-system"/"original-pull-secret" not registered Sep 08 08:10:04.975983 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:04.975955 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlnps\" (UniqueName: \"kubernetes.io/projected/7301238b-b7d9-4101-8f7a-5e442499ca05-kube-api-access-jlnps\") pod \"network-check-target-k45rp\" (UID: \"7301238b-b7d9-4101-8f7a-5e442499ca05\") " pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:10:04.976143 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:04.976116 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 08:10:04.976143 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:04.976135 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 08:10:04.976143 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:04.976145 2585 projected.go:196] Error preparing data for projected volume kube-api-access-jlnps for pod openshift-network-diagnostics/network-check-target-k45rp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 08:10:04.976274 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:04.976196 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7301238b-b7d9-4101-8f7a-5e442499ca05-kube-api-access-jlnps podName:7301238b-b7d9-4101-8f7a-5e442499ca05 nodeName:}" failed. No retries permitted until 2026-09-08 08:10:20.976178547 +0000 UTC m=+34.313818166 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-jlnps" (UniqueName: "kubernetes.io/projected/7301238b-b7d9-4101-8f7a-5e442499ca05-kube-api-access-jlnps") pod "network-check-target-k45rp" (UID: "7301238b-b7d9-4101-8f7a-5e442499ca05") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 08:10:05.169081 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:05.169057 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:10:05.169229 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:05.169096 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:10:05.169229 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:05.169154 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k45rp" podUID="7301238b-b7d9-4101-8f7a-5e442499ca05" Sep 08 08:10:05.169505 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:05.169059 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:10:05.169505 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:05.169459 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mczjr" podUID="dac5eaa3-6c21-4b05-ace5-a13c8344ab6e" Sep 08 08:10:05.169661 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:05.169516 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns5tq" podUID="95940b17-787c-4892-96f0-83cf9501e4d5" Sep 08 08:10:05.304577 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:05.304532 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hhv6k" event={"ID":"f4583a67-e97f-467b-a658-fe2d36c197e2","Type":"ContainerStarted","Data":"b526f14f487b1388d5bb30e0be20b86429ab007382fba498519c922dab0a5acb"} Sep 08 08:10:05.320130 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:05.319213 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-hhv6k" podStartSLOduration=3.917488752 podStartE2EDuration="17.31919506s" totalImagesPullingTime="13.401706308s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 08:09:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:10:05.31900461 +0000 UTC m=+18.656644251" watchObservedRunningTime="2026-09-08 08:10:05.31919506 +0000 UTC m=+18.656834701" Sep 08 08:10:07.040503 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:07.040468 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ffdd8" Sep 08 08:10:07.041344 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:07.041315 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ffdd8" Sep 08 08:10:07.169988 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:07.169957 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:10:07.170155 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:07.170060 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns5tq" podUID="95940b17-787c-4892-96f0-83cf9501e4d5" Sep 08 08:10:07.170318 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:07.170281 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:10:07.170461 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:07.170419 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:10:07.170461 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:07.170422 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k45rp" podUID="7301238b-b7d9-4101-8f7a-5e442499ca05" Sep 08 08:10:07.170566 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:07.170501 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mczjr" podUID="dac5eaa3-6c21-4b05-ace5-a13c8344ab6e" Sep 08 08:10:07.308816 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:07.308733 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ffdd8" Sep 08 08:10:07.309254 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:07.309227 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ffdd8" Sep 08 08:10:09.169591 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:09.169547 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:10:09.169591 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:09.169568 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:10:09.170240 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:09.169560 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:10:09.170240 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:09.169675 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k45rp" podUID="7301238b-b7d9-4101-8f7a-5e442499ca05" Sep 08 08:10:09.170240 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:09.169767 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mczjr" podUID="dac5eaa3-6c21-4b05-ace5-a13c8344ab6e" Sep 08 08:10:09.170240 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:09.169868 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns5tq" podUID="95940b17-787c-4892-96f0-83cf9501e4d5" Sep 08 08:10:11.068657 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:11.068316 2585 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 08 08:10:11.089493 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:11.089409 2585 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-08T08:10:11.068657653Z","UUID":"1284df92-32c5-4b9a-a79e-1889e76f2ad9","Handler":null,"Name":"","Endpoint":""} Sep 08 08:10:11.091302 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:11.091285 2585 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 08 08:10:11.091430 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:11.091315 2585 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 08 08:10:11.169450 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:11.169429 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:10:11.169450 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:11.169441 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:10:11.169556 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:11.169530 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k45rp" podUID="7301238b-b7d9-4101-8f7a-5e442499ca05" Sep 08 08:10:11.169609 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:11.169592 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:10:11.169716 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:11.169694 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns5tq" podUID="95940b17-787c-4892-96f0-83cf9501e4d5" Sep 08 08:10:11.169756 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:11.169584 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mczjr" podUID="dac5eaa3-6c21-4b05-ace5-a13c8344ab6e" Sep 08 08:10:11.318559 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:11.318542 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2xlvk_e00cae51-f771-4039-81ec-54fa2a48e98f/ovn-acl-logging/0.log" Sep 08 08:10:11.318850 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:11.318831 2585 generic.go:356] "Generic (PLEG): container finished" podID="e00cae51-f771-4039-81ec-54fa2a48e98f" containerID="b57061560fe8c9ddeff340e8547c0c1f72ff3b622d60212101f983cb00514cc5" exitCode=1 Sep 08 08:10:11.318927 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:11.318900 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" event={"ID":"e00cae51-f771-4039-81ec-54fa2a48e98f","Type":"ContainerStarted","Data":"21015ddba68dfebf34f8459662df249f6ad58d802caee500bc760e99684d50f1"} Sep 08 08:10:11.318976 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:11.318963 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" event={"ID":"e00cae51-f771-4039-81ec-54fa2a48e98f","Type":"ContainerDied","Data":"b57061560fe8c9ddeff340e8547c0c1f72ff3b622d60212101f983cb00514cc5"} Sep 08 08:10:11.319018 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:11.319004 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" event={"ID":"e00cae51-f771-4039-81ec-54fa2a48e98f","Type":"ContainerStarted","Data":"d07ab53999ecf87a57f6aa508e964dd8213c2b6019791e4883dc9780657111cd"} Sep 08 08:10:11.323847 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:11.323824 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7snsl" event={"ID":"dc7f3943-6396-455a-8432-5129bb96c701","Type":"ContainerStarted","Data":"7bfbf167b2b3cd8f8cefdca84eb2d1960f0ccfb1a75de3b3e523249a45633e3b"} Sep 08 08:10:11.325528 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:11.325508 2585 generic.go:356] "Generic (PLEG): container finished" podID="fcd3e62c-7a33-4211-9b1f-1223aeb19d09" containerID="3adf282404addb99ba3b0f5f57668420b6d228532bb1239fe7d63af8ac7a6435" exitCode=0 Sep 08 08:10:11.325609 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:11.325540 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwkjc" event={"ID":"fcd3e62c-7a33-4211-9b1f-1223aeb19d09","Type":"ContainerDied","Data":"3adf282404addb99ba3b0f5f57668420b6d228532bb1239fe7d63af8ac7a6435"} Sep 08 08:10:12.330094 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:12.329947 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2xlvk_e00cae51-f771-4039-81ec-54fa2a48e98f/ovn-acl-logging/0.log" Sep 08 08:10:12.330473 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:12.330448 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" event={"ID":"e00cae51-f771-4039-81ec-54fa2a48e98f","Type":"ContainerStarted","Data":"7708c76124db4d75301e789cdd36755ef44a6437dead66d3c7618cbe1ca3d9ef"} Sep 08 08:10:12.330544 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:12.330520 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" event={"ID":"e00cae51-f771-4039-81ec-54fa2a48e98f","Type":"ContainerStarted","Data":"8112f3826ffc2b1b78d7bd0338a5d776b481f943581ee2f0aeb147aa0c1e90ad"} Sep 08 08:10:12.330583 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:12.330559 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" event={"ID":"e00cae51-f771-4039-81ec-54fa2a48e98f","Type":"ContainerStarted","Data":"21e110f0e3cc23fca6370d4e4addca98362701394828dbe2a7dcaccffb1ae119"} Sep 08 08:10:12.332274 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:12.332245 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7snsl" event={"ID":"dc7f3943-6396-455a-8432-5129bb96c701","Type":"ContainerStarted","Data":"75ff5df563f2cc4ab85e37be8e80a54dfe9a575ecbcb092853b9bfce18417848"} Sep 08 08:10:12.334875 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:12.334848 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwkjc" event={"ID":"fcd3e62c-7a33-4211-9b1f-1223aeb19d09","Type":"ContainerStarted","Data":"6afe44764c77cbc61c43ea8327f701955c42afba6bf9756bff308f3c174043dd"} Sep 08 08:10:12.350313 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:12.350275 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7snsl" podStartSLOduration=3.784674189 podStartE2EDuration="25.350262028s" totalImagesPullingTime="21.565587839s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 08:09:47 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:10:12.349914519 +0000 UTC m=+25.687554160" watchObservedRunningTime="2026-09-08 08:10:12.350262028 +0000 UTC m=+25.687901668" Sep 08 08:10:13.169002 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:13.168973 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:10:13.169002 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:13.168997 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:10:13.169231 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:13.168996 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:10:13.169231 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:13.169074 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k45rp" podUID="7301238b-b7d9-4101-8f7a-5e442499ca05" Sep 08 08:10:13.169231 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:13.169147 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mczjr" podUID="dac5eaa3-6c21-4b05-ace5-a13c8344ab6e" Sep 08 08:10:13.169231 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:13.169209 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns5tq" podUID="95940b17-787c-4892-96f0-83cf9501e4d5" Sep 08 08:10:13.338861 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:13.338838 2585 generic.go:356] "Generic (PLEG): container finished" podID="fcd3e62c-7a33-4211-9b1f-1223aeb19d09" containerID="6afe44764c77cbc61c43ea8327f701955c42afba6bf9756bff308f3c174043dd" exitCode=0 Sep 08 08:10:13.339218 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:13.338930 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwkjc" event={"ID":"fcd3e62c-7a33-4211-9b1f-1223aeb19d09","Type":"ContainerDied","Data":"6afe44764c77cbc61c43ea8327f701955c42afba6bf9756bff308f3c174043dd"} Sep 08 08:10:14.344071 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:14.344016 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2xlvk_e00cae51-f771-4039-81ec-54fa2a48e98f/ovn-acl-logging/0.log" Sep 08 08:10:14.344518 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:14.344425 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" event={"ID":"e00cae51-f771-4039-81ec-54fa2a48e98f","Type":"ContainerStarted","Data":"7a1d0804c53358ec529c8c7198228654871f3870a02cd5df6afba77c9b6b03fc"} Sep 08 08:10:14.346785 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:14.346757 2585 generic.go:356] "Generic (PLEG): container finished" podID="fcd3e62c-7a33-4211-9b1f-1223aeb19d09" containerID="8801a95f745472f831b99ad9a662ee8af996a91a2b985557020f76fd4a5d0c25" exitCode=0 Sep 08 08:10:14.346866 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:14.346792 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwkjc" event={"ID":"fcd3e62c-7a33-4211-9b1f-1223aeb19d09","Type":"ContainerDied","Data":"8801a95f745472f831b99ad9a662ee8af996a91a2b985557020f76fd4a5d0c25"} Sep 08 08:10:15.169237 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:15.169205 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:10:15.169433 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:15.169204 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:10:15.169433 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:15.169340 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns5tq" podUID="95940b17-787c-4892-96f0-83cf9501e4d5" Sep 08 08:10:15.169433 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:15.169360 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:10:15.169609 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:15.169466 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k45rp" podUID="7301238b-b7d9-4101-8f7a-5e442499ca05" Sep 08 08:10:15.169609 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:15.169545 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mczjr" podUID="dac5eaa3-6c21-4b05-ace5-a13c8344ab6e" Sep 08 08:10:16.356885 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:16.356827 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2xlvk_e00cae51-f771-4039-81ec-54fa2a48e98f/ovn-acl-logging/0.log" Sep 08 08:10:16.357778 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:16.357237 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" event={"ID":"e00cae51-f771-4039-81ec-54fa2a48e98f","Type":"ContainerStarted","Data":"7bbb1e8a32275f705eaff393016923af3899765d1c98029c03de35d2eee9e41a"} Sep 08 08:10:16.357778 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:16.357554 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:10:16.357778 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:16.357760 2585 scope.go:117] "RemoveContainer" containerID="b57061560fe8c9ddeff340e8547c0c1f72ff3b622d60212101f983cb00514cc5" Sep 08 08:10:16.376689 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:16.376514 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:10:17.170616 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:17.170578 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:10:17.170811 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:17.170665 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns5tq" podUID="95940b17-787c-4892-96f0-83cf9501e4d5" Sep 08 08:10:17.170811 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:17.170767 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:10:17.170935 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:17.170847 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k45rp" podUID="7301238b-b7d9-4101-8f7a-5e442499ca05" Sep 08 08:10:17.170935 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:17.170896 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:10:17.171034 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:17.170962 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mczjr" podUID="dac5eaa3-6c21-4b05-ace5-a13c8344ab6e" Sep 08 08:10:17.364449 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:17.364416 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2xlvk_e00cae51-f771-4039-81ec-54fa2a48e98f/ovn-acl-logging/0.log" Sep 08 08:10:17.364895 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:17.364874 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" event={"ID":"e00cae51-f771-4039-81ec-54fa2a48e98f","Type":"ContainerStarted","Data":"dd88bb4f03db1bcf26ecfe59c42fe067e0640182fd821590b1232978bd5d4e23"} Sep 08 08:10:17.365470 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:17.365106 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:10:17.365470 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:17.365179 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:10:17.383774 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:17.383749 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:10:17.396491 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:17.396445 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" podStartSLOduration=8.734803823 podStartE2EDuration="29.39642837s" totalImagesPullingTime="20.661624547s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 08:09:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:10:17.395905772 +0000 UTC m=+30.733545413" watchObservedRunningTime="2026-09-08 08:10:17.39642837 +0000 UTC m=+30.734068016" Sep 08 08:10:19.169037 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:19.169006 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:10:19.169435 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:19.169106 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k45rp" podUID="7301238b-b7d9-4101-8f7a-5e442499ca05" Sep 08 08:10:19.169435 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:19.169016 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:10:19.169435 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:19.169196 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mczjr" podUID="dac5eaa3-6c21-4b05-ace5-a13c8344ab6e" Sep 08 08:10:19.169435 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:19.169221 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:10:19.169435 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:19.169293 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns5tq" podUID="95940b17-787c-4892-96f0-83cf9501e4d5" Sep 08 08:10:19.371442 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:19.371410 2585 generic.go:356] "Generic (PLEG): container finished" podID="fcd3e62c-7a33-4211-9b1f-1223aeb19d09" containerID="bd7d1b7a7c4663fcd526a0dd2abcb6c76912a3bc04d50a43a18e5b8ead91d087" exitCode=0 Sep 08 08:10:19.371560 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:19.371483 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwkjc" event={"ID":"fcd3e62c-7a33-4211-9b1f-1223aeb19d09","Type":"ContainerDied","Data":"bd7d1b7a7c4663fcd526a0dd2abcb6c76912a3bc04d50a43a18e5b8ead91d087"} Sep 08 08:10:20.378669 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:20.378634 2585 generic.go:356] "Generic (PLEG): container finished" podID="fcd3e62c-7a33-4211-9b1f-1223aeb19d09" containerID="2bb7e6affa86df0183a47159e3640ecd33625ccd73fe05a59fe9d4eeb5ee8664" exitCode=0 Sep 08 08:10:20.379066 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:20.378674 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwkjc" event={"ID":"fcd3e62c-7a33-4211-9b1f-1223aeb19d09","Type":"ContainerDied","Data":"2bb7e6affa86df0183a47159e3640ecd33625ccd73fe05a59fe9d4eeb5ee8664"} Sep 08 08:10:20.789830 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:20.789796 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95940b17-787c-4892-96f0-83cf9501e4d5-metrics-certs\") pod \"network-metrics-daemon-ns5tq\" (UID: \"95940b17-787c-4892-96f0-83cf9501e4d5\") " pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:10:20.789943 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:20.789925 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 08:10:20.789988 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:20.789980 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95940b17-787c-4892-96f0-83cf9501e4d5-metrics-certs podName:95940b17-787c-4892-96f0-83cf9501e4d5 nodeName:}" failed. No retries permitted until 2026-09-08 08:10:52.789964122 +0000 UTC m=+66.127603741 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/95940b17-787c-4892-96f0-83cf9501e4d5-metrics-certs") pod "network-metrics-daemon-ns5tq" (UID: "95940b17-787c-4892-96f0-83cf9501e4d5") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 08:10:20.890702 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:20.890675 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dac5eaa3-6c21-4b05-ace5-a13c8344ab6e-original-pull-secret\") pod \"global-pull-secret-syncer-mczjr\" (UID: \"dac5eaa3-6c21-4b05-ace5-a13c8344ab6e\") " pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:10:20.890797 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:20.890768 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 08:10:20.890829 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:20.890810 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dac5eaa3-6c21-4b05-ace5-a13c8344ab6e-original-pull-secret podName:dac5eaa3-6c21-4b05-ace5-a13c8344ab6e nodeName:}" failed. No retries permitted until 2026-09-08 08:10:52.890799015 +0000 UTC m=+66.228438634 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dac5eaa3-6c21-4b05-ace5-a13c8344ab6e-original-pull-secret") pod "global-pull-secret-syncer-mczjr" (UID: "dac5eaa3-6c21-4b05-ace5-a13c8344ab6e") : object "kube-system"/"original-pull-secret" not registered Sep 08 08:10:20.991342 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:20.991320 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlnps\" (UniqueName: \"kubernetes.io/projected/7301238b-b7d9-4101-8f7a-5e442499ca05-kube-api-access-jlnps\") pod \"network-check-target-k45rp\" (UID: \"7301238b-b7d9-4101-8f7a-5e442499ca05\") " pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:10:20.991491 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:20.991475 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 08:10:20.991548 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:20.991497 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 08:10:20.991548 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:20.991509 2585 projected.go:196] Error preparing data for projected volume kube-api-access-jlnps for pod openshift-network-diagnostics/network-check-target-k45rp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 08:10:20.991621 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:20.991549 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7301238b-b7d9-4101-8f7a-5e442499ca05-kube-api-access-jlnps podName:7301238b-b7d9-4101-8f7a-5e442499ca05 nodeName:}" failed. No retries permitted until 2026-09-08 08:10:52.9915376 +0000 UTC m=+66.329177223 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-jlnps" (UniqueName: "kubernetes.io/projected/7301238b-b7d9-4101-8f7a-5e442499ca05-kube-api-access-jlnps") pod "network-check-target-k45rp" (UID: "7301238b-b7d9-4101-8f7a-5e442499ca05") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 08:10:21.170084 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:21.170058 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:10:21.170237 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:21.170149 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns5tq" podUID="95940b17-787c-4892-96f0-83cf9501e4d5" Sep 08 08:10:21.170237 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:21.170208 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:10:21.170358 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:21.170320 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k45rp" podUID="7301238b-b7d9-4101-8f7a-5e442499ca05" Sep 08 08:10:21.170422 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:21.170353 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:10:21.170493 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:21.170473 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mczjr" podUID="dac5eaa3-6c21-4b05-ace5-a13c8344ab6e" Sep 08 08:10:21.386200 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:21.386171 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwkjc" event={"ID":"fcd3e62c-7a33-4211-9b1f-1223aeb19d09","Type":"ContainerStarted","Data":"fe01b353d3913424f931789ae6d58564e08b4cb2af1f19803ae1f5a6c1c676e9"} Sep 08 08:10:21.409516 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:21.409475 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zwkjc" podStartSLOduration=7.419892824 podStartE2EDuration="33.409460698s" totalImagesPullingTime="25.989567874s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 08:09:48 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:10:21.408928361 +0000 UTC m=+34.746568002" watchObservedRunningTime="2026-09-08 08:10:21.409460698 +0000 UTC m=+34.747100339" Sep 08 08:10:23.169288 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:23.169257 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:10:23.169288 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:23.169271 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:10:23.169823 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:23.169262 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:10:23.169823 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:23.169364 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k45rp" podUID="7301238b-b7d9-4101-8f7a-5e442499ca05" Sep 08 08:10:23.169823 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:23.169436 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns5tq" podUID="95940b17-787c-4892-96f0-83cf9501e4d5" Sep 08 08:10:23.169823 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:23.169516 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mczjr" podUID="dac5eaa3-6c21-4b05-ace5-a13c8344ab6e" Sep 08 08:10:25.169942 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:25.169910 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:10:25.169942 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:25.169925 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:10:25.170441 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:25.169915 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:10:25.170441 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:25.170004 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns5tq" podUID="95940b17-787c-4892-96f0-83cf9501e4d5" Sep 08 08:10:25.170441 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:25.170154 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k45rp" podUID="7301238b-b7d9-4101-8f7a-5e442499ca05" Sep 08 08:10:25.170441 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:25.170189 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mczjr" podUID="dac5eaa3-6c21-4b05-ace5-a13c8344ab6e" Sep 08 08:10:27.170326 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:27.170302 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:10:27.170677 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:27.170396 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns5tq" podUID="95940b17-787c-4892-96f0-83cf9501e4d5" Sep 08 08:10:27.170677 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:27.170458 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:10:27.170677 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:27.170533 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:10:27.170677 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:27.170584 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k45rp" podUID="7301238b-b7d9-4101-8f7a-5e442499ca05" Sep 08 08:10:27.170677 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:27.170590 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mczjr" podUID="dac5eaa3-6c21-4b05-ace5-a13c8344ab6e" Sep 08 08:10:29.169414 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:29.169366 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:10:29.169761 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:29.169425 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:10:29.169761 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:29.169366 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:10:29.169761 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:29.169484 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k45rp" podUID="7301238b-b7d9-4101-8f7a-5e442499ca05" Sep 08 08:10:29.169761 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:29.169585 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns5tq" podUID="95940b17-787c-4892-96f0-83cf9501e4d5" Sep 08 08:10:29.169761 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:29.169661 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mczjr" podUID="dac5eaa3-6c21-4b05-ace5-a13c8344ab6e" Sep 08 08:10:31.169532 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:31.169504 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:10:31.169906 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:31.169514 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:10:31.169906 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:31.169606 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns5tq" podUID="95940b17-787c-4892-96f0-83cf9501e4d5" Sep 08 08:10:31.169906 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:31.169514 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:10:31.169906 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:31.169674 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k45rp" podUID="7301238b-b7d9-4101-8f7a-5e442499ca05" Sep 08 08:10:31.169906 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:31.169736 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mczjr" podUID="dac5eaa3-6c21-4b05-ace5-a13c8344ab6e" Sep 08 08:10:31.901665 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:31.901481 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-k45rp"] Sep 08 08:10:31.901795 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:31.901776 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:10:31.901889 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:31.901866 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k45rp" podUID="7301238b-b7d9-4101-8f7a-5e442499ca05" Sep 08 08:10:31.904778 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:31.904732 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mczjr"] Sep 08 08:10:31.904911 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:31.904887 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:10:31.905025 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:31.905004 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mczjr" podUID="dac5eaa3-6c21-4b05-ace5-a13c8344ab6e" Sep 08 08:10:31.905342 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:31.905317 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ns5tq"] Sep 08 08:10:31.905467 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:31.905437 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:10:31.905565 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:31.905542 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns5tq" podUID="95940b17-787c-4892-96f0-83cf9501e4d5" Sep 08 08:10:33.169807 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:33.169780 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:10:33.170216 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:33.169870 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k45rp" podUID="7301238b-b7d9-4101-8f7a-5e442499ca05" Sep 08 08:10:33.170216 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:33.169958 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:10:33.170216 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:33.170065 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mczjr" podUID="dac5eaa3-6c21-4b05-ace5-a13c8344ab6e" Sep 08 08:10:34.169854 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:34.169829 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:10:34.170358 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:34.169927 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns5tq" podUID="95940b17-787c-4892-96f0-83cf9501e4d5" Sep 08 08:10:35.169536 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.169509 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:10:35.169536 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.169525 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:10:35.169785 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:35.169627 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k45rp" podUID="7301238b-b7d9-4101-8f7a-5e442499ca05" Sep 08 08:10:35.169785 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:35.169710 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mczjr" podUID="dac5eaa3-6c21-4b05-ace5-a13c8344ab6e" Sep 08 08:10:35.474989 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.474930 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-1.ec2.internal" event="NodeReady" Sep 08 08:10:35.475375 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.475023 2585 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 08 08:10:35.518902 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.518867 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-rdr6d"] Sep 08 08:10:35.520830 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.520809 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rdr6d" Sep 08 08:10:35.521726 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.521702 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-jbjgx"] Sep 08 08:10:35.523097 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.523076 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 08 08:10:35.523097 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.523088 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-phzhh\"" Sep 08 08:10:35.523233 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.523088 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 08 08:10:35.523333 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.523314 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jbjgx" Sep 08 08:10:35.525551 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.525528 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 08 08:10:35.525638 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.525561 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 08 08:10:35.526013 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.525998 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 08 08:10:35.527732 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.527717 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-t5svz\"" Sep 08 08:10:35.531893 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.531872 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rdr6d"] Sep 08 08:10:35.535109 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.534937 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jbjgx"] Sep 08 08:10:35.542159 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.542140 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-v5pd2"] Sep 08 08:10:35.543932 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.543912 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-v5pd2" Sep 08 08:10:35.546165 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.546150 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 08 08:10:35.546281 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.546216 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 08 08:10:35.546281 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.546265 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 08 08:10:35.546410 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.546322 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-w7frm\"" Sep 08 08:10:35.546410 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.546269 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 08 08:10:35.557259 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.557241 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-v5pd2"] Sep 08 08:10:35.694543 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.694519 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28v6p\" (UniqueName: \"kubernetes.io/projected/9f709c93-d0b7-4369-98aa-5e912759dead-kube-api-access-28v6p\") pod \"dns-default-rdr6d\" (UID: \"9f709c93-d0b7-4369-98aa-5e912759dead\") " pod="openshift-dns/dns-default-rdr6d" Sep 08 08:10:35.694698 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.694560 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8fd236ec-d3a6-4c13-b01f-6035a2d96c55-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v5pd2\" (UID: \"8fd236ec-d3a6-4c13-b01f-6035a2d96c55\") " pod="openshift-insights/insights-runtime-extractor-v5pd2" Sep 08 08:10:35.694698 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.694581 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9f709c93-d0b7-4369-98aa-5e912759dead-config-volume\") pod \"dns-default-rdr6d\" (UID: \"9f709c93-d0b7-4369-98aa-5e912759dead\") " pod="openshift-dns/dns-default-rdr6d" Sep 08 08:10:35.694698 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.694597 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9f709c93-d0b7-4369-98aa-5e912759dead-metrics-tls\") pod \"dns-default-rdr6d\" (UID: \"9f709c93-d0b7-4369-98aa-5e912759dead\") " pod="openshift-dns/dns-default-rdr6d" Sep 08 08:10:35.694698 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.694620 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f0afdfcb-a4d0-41e1-8928-f251568daaef-cert\") pod \"ingress-canary-jbjgx\" (UID: \"f0afdfcb-a4d0-41e1-8928-f251568daaef\") " pod="openshift-ingress-canary/ingress-canary-jbjgx" Sep 08 08:10:35.694698 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.694681 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqmhz\" (UniqueName: \"kubernetes.io/projected/8fd236ec-d3a6-4c13-b01f-6035a2d96c55-kube-api-access-kqmhz\") pod \"insights-runtime-extractor-v5pd2\" (UID: \"8fd236ec-d3a6-4c13-b01f-6035a2d96c55\") " pod="openshift-insights/insights-runtime-extractor-v5pd2" Sep 08 08:10:35.694914 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.694720 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qltl6\" (UniqueName: \"kubernetes.io/projected/f0afdfcb-a4d0-41e1-8928-f251568daaef-kube-api-access-qltl6\") pod \"ingress-canary-jbjgx\" (UID: \"f0afdfcb-a4d0-41e1-8928-f251568daaef\") " pod="openshift-ingress-canary/ingress-canary-jbjgx" Sep 08 08:10:35.694914 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.694766 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8fd236ec-d3a6-4c13-b01f-6035a2d96c55-crio-socket\") pod \"insights-runtime-extractor-v5pd2\" (UID: \"8fd236ec-d3a6-4c13-b01f-6035a2d96c55\") " pod="openshift-insights/insights-runtime-extractor-v5pd2" Sep 08 08:10:35.694914 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.694805 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8fd236ec-d3a6-4c13-b01f-6035a2d96c55-data-volume\") pod \"insights-runtime-extractor-v5pd2\" (UID: \"8fd236ec-d3a6-4c13-b01f-6035a2d96c55\") " pod="openshift-insights/insights-runtime-extractor-v5pd2" Sep 08 08:10:35.694914 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.694840 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8fd236ec-d3a6-4c13-b01f-6035a2d96c55-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v5pd2\" (UID: \"8fd236ec-d3a6-4c13-b01f-6035a2d96c55\") " pod="openshift-insights/insights-runtime-extractor-v5pd2" Sep 08 08:10:35.694914 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.694857 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f709c93-d0b7-4369-98aa-5e912759dead-tmp-dir\") pod \"dns-default-rdr6d\" (UID: \"9f709c93-d0b7-4369-98aa-5e912759dead\") " pod="openshift-dns/dns-default-rdr6d" Sep 08 08:10:35.795825 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.795806 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8fd236ec-d3a6-4c13-b01f-6035a2d96c55-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v5pd2\" (UID: \"8fd236ec-d3a6-4c13-b01f-6035a2d96c55\") " pod="openshift-insights/insights-runtime-extractor-v5pd2" Sep 08 08:10:35.795915 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.795833 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f709c93-d0b7-4369-98aa-5e912759dead-tmp-dir\") pod \"dns-default-rdr6d\" (UID: \"9f709c93-d0b7-4369-98aa-5e912759dead\") " pod="openshift-dns/dns-default-rdr6d" Sep 08 08:10:35.795915 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.795862 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28v6p\" (UniqueName: \"kubernetes.io/projected/9f709c93-d0b7-4369-98aa-5e912759dead-kube-api-access-28v6p\") pod \"dns-default-rdr6d\" (UID: \"9f709c93-d0b7-4369-98aa-5e912759dead\") " pod="openshift-dns/dns-default-rdr6d" Sep 08 08:10:35.795915 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.795887 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8fd236ec-d3a6-4c13-b01f-6035a2d96c55-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v5pd2\" (UID: \"8fd236ec-d3a6-4c13-b01f-6035a2d96c55\") " pod="openshift-insights/insights-runtime-extractor-v5pd2" Sep 08 08:10:35.795915 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.795905 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9f709c93-d0b7-4369-98aa-5e912759dead-config-volume\") pod \"dns-default-rdr6d\" (UID: \"9f709c93-d0b7-4369-98aa-5e912759dead\") " pod="openshift-dns/dns-default-rdr6d" Sep 08 08:10:35.796108 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.796018 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9f709c93-d0b7-4369-98aa-5e912759dead-metrics-tls\") pod \"dns-default-rdr6d\" (UID: \"9f709c93-d0b7-4369-98aa-5e912759dead\") " pod="openshift-dns/dns-default-rdr6d" Sep 08 08:10:35.796108 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.796052 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f0afdfcb-a4d0-41e1-8928-f251568daaef-cert\") pod \"ingress-canary-jbjgx\" (UID: \"f0afdfcb-a4d0-41e1-8928-f251568daaef\") " pod="openshift-ingress-canary/ingress-canary-jbjgx" Sep 08 08:10:35.796203 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.796168 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqmhz\" (UniqueName: \"kubernetes.io/projected/8fd236ec-d3a6-4c13-b01f-6035a2d96c55-kube-api-access-kqmhz\") pod \"insights-runtime-extractor-v5pd2\" (UID: \"8fd236ec-d3a6-4c13-b01f-6035a2d96c55\") " pod="openshift-insights/insights-runtime-extractor-v5pd2" Sep 08 08:10:35.796260 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.796212 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qltl6\" (UniqueName: \"kubernetes.io/projected/f0afdfcb-a4d0-41e1-8928-f251568daaef-kube-api-access-qltl6\") pod \"ingress-canary-jbjgx\" (UID: \"f0afdfcb-a4d0-41e1-8928-f251568daaef\") " pod="openshift-ingress-canary/ingress-canary-jbjgx" Sep 08 08:10:35.796260 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.796208 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f709c93-d0b7-4369-98aa-5e912759dead-tmp-dir\") pod \"dns-default-rdr6d\" (UID: \"9f709c93-d0b7-4369-98aa-5e912759dead\") " pod="openshift-dns/dns-default-rdr6d" Sep 08 08:10:35.796260 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.796250 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8fd236ec-d3a6-4c13-b01f-6035a2d96c55-crio-socket\") pod \"insights-runtime-extractor-v5pd2\" (UID: \"8fd236ec-d3a6-4c13-b01f-6035a2d96c55\") " pod="openshift-insights/insights-runtime-extractor-v5pd2" Sep 08 08:10:35.796423 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.796285 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8fd236ec-d3a6-4c13-b01f-6035a2d96c55-data-volume\") pod \"insights-runtime-extractor-v5pd2\" (UID: \"8fd236ec-d3a6-4c13-b01f-6035a2d96c55\") " pod="openshift-insights/insights-runtime-extractor-v5pd2" Sep 08 08:10:35.796485 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.796428 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8fd236ec-d3a6-4c13-b01f-6035a2d96c55-crio-socket\") pod \"insights-runtime-extractor-v5pd2\" (UID: \"8fd236ec-d3a6-4c13-b01f-6035a2d96c55\") " pod="openshift-insights/insights-runtime-extractor-v5pd2" Sep 08 08:10:35.796577 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.796559 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8fd236ec-d3a6-4c13-b01f-6035a2d96c55-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v5pd2\" (UID: \"8fd236ec-d3a6-4c13-b01f-6035a2d96c55\") " pod="openshift-insights/insights-runtime-extractor-v5pd2" Sep 08 08:10:35.796628 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.796575 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8fd236ec-d3a6-4c13-b01f-6035a2d96c55-data-volume\") pod \"insights-runtime-extractor-v5pd2\" (UID: \"8fd236ec-d3a6-4c13-b01f-6035a2d96c55\") " pod="openshift-insights/insights-runtime-extractor-v5pd2" Sep 08 08:10:35.796628 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.796590 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9f709c93-d0b7-4369-98aa-5e912759dead-config-volume\") pod \"dns-default-rdr6d\" (UID: \"9f709c93-d0b7-4369-98aa-5e912759dead\") " pod="openshift-dns/dns-default-rdr6d" Sep 08 08:10:35.799880 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.799858 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9f709c93-d0b7-4369-98aa-5e912759dead-metrics-tls\") pod \"dns-default-rdr6d\" (UID: \"9f709c93-d0b7-4369-98aa-5e912759dead\") " pod="openshift-dns/dns-default-rdr6d" Sep 08 08:10:35.799968 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.799885 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8fd236ec-d3a6-4c13-b01f-6035a2d96c55-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v5pd2\" (UID: \"8fd236ec-d3a6-4c13-b01f-6035a2d96c55\") " pod="openshift-insights/insights-runtime-extractor-v5pd2" Sep 08 08:10:35.799968 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.799952 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f0afdfcb-a4d0-41e1-8928-f251568daaef-cert\") pod \"ingress-canary-jbjgx\" (UID: \"f0afdfcb-a4d0-41e1-8928-f251568daaef\") " pod="openshift-ingress-canary/ingress-canary-jbjgx" Sep 08 08:10:35.806935 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.806913 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28v6p\" (UniqueName: \"kubernetes.io/projected/9f709c93-d0b7-4369-98aa-5e912759dead-kube-api-access-28v6p\") pod \"dns-default-rdr6d\" (UID: \"9f709c93-d0b7-4369-98aa-5e912759dead\") " pod="openshift-dns/dns-default-rdr6d" Sep 08 08:10:35.807540 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.807519 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqmhz\" (UniqueName: \"kubernetes.io/projected/8fd236ec-d3a6-4c13-b01f-6035a2d96c55-kube-api-access-kqmhz\") pod \"insights-runtime-extractor-v5pd2\" (UID: \"8fd236ec-d3a6-4c13-b01f-6035a2d96c55\") " pod="openshift-insights/insights-runtime-extractor-v5pd2" Sep 08 08:10:35.808066 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.808051 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qltl6\" (UniqueName: \"kubernetes.io/projected/f0afdfcb-a4d0-41e1-8928-f251568daaef-kube-api-access-qltl6\") pod \"ingress-canary-jbjgx\" (UID: \"f0afdfcb-a4d0-41e1-8928-f251568daaef\") " pod="openshift-ingress-canary/ingress-canary-jbjgx" Sep 08 08:10:35.831459 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.831440 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rdr6d" Sep 08 08:10:35.837855 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.837839 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jbjgx" Sep 08 08:10:35.851837 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:35.851815 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-v5pd2" Sep 08 08:10:36.117719 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:36.117213 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jbjgx"] Sep 08 08:10:36.122852 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:10:36.122822 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0afdfcb_a4d0_41e1_8928_f251568daaef.slice/crio-005a58b6541cc4a758b09539ad6270afd9fb655f531cc91d7e13fe1e7b3f9fc7 WatchSource:0}: Error finding container 005a58b6541cc4a758b09539ad6270afd9fb655f531cc91d7e13fe1e7b3f9fc7: Status 404 returned error can't find the container with id 005a58b6541cc4a758b09539ad6270afd9fb655f531cc91d7e13fe1e7b3f9fc7 Sep 08 08:10:36.123063 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:36.123037 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rdr6d"] Sep 08 08:10:36.126063 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:10:36.126042 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f709c93_d0b7_4369_98aa_5e912759dead.slice/crio-3b2a8ff83514136bf7742a570ce6c9bc4542a29ed92d1677f7be3f8a7e7de381 WatchSource:0}: Error finding container 3b2a8ff83514136bf7742a570ce6c9bc4542a29ed92d1677f7be3f8a7e7de381: Status 404 returned error can't find the container with id 3b2a8ff83514136bf7742a570ce6c9bc4542a29ed92d1677f7be3f8a7e7de381 Sep 08 08:10:36.129835 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:36.129814 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-v5pd2"] Sep 08 08:10:36.132902 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:10:36.132884 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fd236ec_d3a6_4c13_b01f_6035a2d96c55.slice/crio-f58ba363aa5e25b572ea22085f3f047b1e134dbf446ffb99f1ffd6759b6834b8 WatchSource:0}: Error finding container f58ba363aa5e25b572ea22085f3f047b1e134dbf446ffb99f1ffd6759b6834b8: Status 404 returned error can't find the container with id f58ba363aa5e25b572ea22085f3f047b1e134dbf446ffb99f1ffd6759b6834b8 Sep 08 08:10:36.169190 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:36.169171 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:10:36.171601 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:36.171584 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-n4fwr\"" Sep 08 08:10:36.171693 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:36.171584 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 08 08:10:36.417915 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:36.417888 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jbjgx" event={"ID":"f0afdfcb-a4d0-41e1-8928-f251568daaef","Type":"ContainerStarted","Data":"005a58b6541cc4a758b09539ad6270afd9fb655f531cc91d7e13fe1e7b3f9fc7"} Sep 08 08:10:36.418939 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:36.418909 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rdr6d" event={"ID":"9f709c93-d0b7-4369-98aa-5e912759dead","Type":"ContainerStarted","Data":"3b2a8ff83514136bf7742a570ce6c9bc4542a29ed92d1677f7be3f8a7e7de381"} Sep 08 08:10:36.420339 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:36.420312 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v5pd2" event={"ID":"8fd236ec-d3a6-4c13-b01f-6035a2d96c55","Type":"ContainerStarted","Data":"36ad8fc0cf0257c192c7d488bfe9c86eacfc7ca62307e97ff51dbc227c743e52"} Sep 08 08:10:36.420427 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:36.420352 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v5pd2" event={"ID":"8fd236ec-d3a6-4c13-b01f-6035a2d96c55","Type":"ContainerStarted","Data":"f58ba363aa5e25b572ea22085f3f047b1e134dbf446ffb99f1ffd6759b6834b8"} Sep 08 08:10:37.171956 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:37.171916 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:10:37.172605 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:37.172589 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:10:37.175299 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:37.175274 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 08 08:10:37.175909 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:37.175889 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vr4wt\"" Sep 08 08:10:37.175909 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:37.175903 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 08 08:10:37.176071 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:37.175907 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 08 08:10:37.425316 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:37.425190 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v5pd2" event={"ID":"8fd236ec-d3a6-4c13-b01f-6035a2d96c55","Type":"ContainerStarted","Data":"1ffd7c8ea897dea886f13a358e94bd2f86b5195c89ce352003f1efe1931608d7"} Sep 08 08:10:39.436242 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:39.436161 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jbjgx" event={"ID":"f0afdfcb-a4d0-41e1-8928-f251568daaef","Type":"ContainerStarted","Data":"56a262807b3c970c9c63b3ef0e7b816b171f9336ae1fde92cefb8cf1f9dc5d5e"} Sep 08 08:10:39.438205 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:39.438182 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rdr6d" event={"ID":"9f709c93-d0b7-4369-98aa-5e912759dead","Type":"ContainerStarted","Data":"8c4ca9084fa07c3460ebb469162863f63e9a16044a560f339a926065f1a37a79"} Sep 08 08:10:39.438320 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:39.438227 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rdr6d" event={"ID":"9f709c93-d0b7-4369-98aa-5e912759dead","Type":"ContainerStarted","Data":"1dea61acd0330e268ba16ef8288f76b5d8b5535a7c6b40b1426ccdfa6e953e0b"} Sep 08 08:10:39.438320 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:39.438276 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-rdr6d" Sep 08 08:10:39.452192 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:39.452148 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-jbjgx" podStartSLOduration=1.646058285 podStartE2EDuration="4.452132957s" totalImagesPullingTime="2.806074672s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 08:10:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:10:39.451555704 +0000 UTC m=+52.789195346" watchObservedRunningTime="2026-09-08 08:10:39.452132957 +0000 UTC m=+52.789772611" Sep 08 08:10:39.467857 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:39.467806 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-rdr6d" podStartSLOduration=1.673101053 podStartE2EDuration="4.467785969s" totalImagesPullingTime="2.794684916s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 08:10:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:10:39.467240328 +0000 UTC m=+52.804879970" watchObservedRunningTime="2026-09-08 08:10:39.467785969 +0000 UTC m=+52.805425613" Sep 08 08:10:40.443989 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:40.443904 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v5pd2" event={"ID":"8fd236ec-d3a6-4c13-b01f-6035a2d96c55","Type":"ContainerStarted","Data":"2ae8c3df7b80714d4e3e8e54206624153df21a4a5e23a56869f900aa2749a49d"} Sep 08 08:10:40.463255 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:40.463213 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-v5pd2" podStartSLOduration=1.5855850280000001 podStartE2EDuration="5.463197789s" totalImagesPullingTime="3.877612761s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 08:10:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:10:40.46223555 +0000 UTC m=+53.799875191" watchObservedRunningTime="2026-09-08 08:10:40.463197789 +0000 UTC m=+53.800837429" Sep 08 08:10:43.007055 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:43.007022 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-tvhls"] Sep 08 08:10:43.022655 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:43.022633 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-tvhls"] Sep 08 08:10:43.022783 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:43.022727 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-tvhls" Sep 08 08:10:43.025171 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:43.025143 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 08 08:10:43.025299 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:43.025150 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 08 08:10:43.025299 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:43.025173 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 08 08:10:43.026594 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:43.026346 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 08 08:10:43.026702 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:43.026664 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 08 08:10:43.029441 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:43.026838 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-lgqdn\"" Sep 08 08:10:43.049332 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:43.049305 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6333538e-9193-4692-946f-8f44c4869ceb-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-tvhls\" (UID: \"6333538e-9193-4692-946f-8f44c4869ceb\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-tvhls" Sep 08 08:10:43.049449 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:43.049338 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpswp\" (UniqueName: \"kubernetes.io/projected/6333538e-9193-4692-946f-8f44c4869ceb-kube-api-access-lpswp\") pod \"prometheus-operator-6f4946db56-tvhls\" (UID: \"6333538e-9193-4692-946f-8f44c4869ceb\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-tvhls" Sep 08 08:10:43.049449 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:43.049390 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6333538e-9193-4692-946f-8f44c4869ceb-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-tvhls\" (UID: \"6333538e-9193-4692-946f-8f44c4869ceb\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-tvhls" Sep 08 08:10:43.049449 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:43.049437 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/6333538e-9193-4692-946f-8f44c4869ceb-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-tvhls\" (UID: \"6333538e-9193-4692-946f-8f44c4869ceb\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-tvhls" Sep 08 08:10:43.149980 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:43.149946 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6333538e-9193-4692-946f-8f44c4869ceb-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-tvhls\" (UID: \"6333538e-9193-4692-946f-8f44c4869ceb\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-tvhls" Sep 08 08:10:43.150165 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:43.149989 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpswp\" (UniqueName: \"kubernetes.io/projected/6333538e-9193-4692-946f-8f44c4869ceb-kube-api-access-lpswp\") pod \"prometheus-operator-6f4946db56-tvhls\" (UID: \"6333538e-9193-4692-946f-8f44c4869ceb\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-tvhls" Sep 08 08:10:43.150165 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:43.150136 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6333538e-9193-4692-946f-8f44c4869ceb-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-tvhls\" (UID: \"6333538e-9193-4692-946f-8f44c4869ceb\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-tvhls" Sep 08 08:10:43.150251 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:43.150177 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/6333538e-9193-4692-946f-8f44c4869ceb-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-tvhls\" (UID: \"6333538e-9193-4692-946f-8f44c4869ceb\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-tvhls" Sep 08 08:10:43.150298 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:43.150283 2585 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Sep 08 08:10:43.150353 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:43.150343 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6333538e-9193-4692-946f-8f44c4869ceb-prometheus-operator-tls podName:6333538e-9193-4692-946f-8f44c4869ceb nodeName:}" failed. No retries permitted until 2026-09-08 08:10:43.650326253 +0000 UTC m=+56.987965872 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/6333538e-9193-4692-946f-8f44c4869ceb-prometheus-operator-tls") pod "prometheus-operator-6f4946db56-tvhls" (UID: "6333538e-9193-4692-946f-8f44c4869ceb") : secret "prometheus-operator-tls" not found Sep 08 08:10:43.150709 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:43.150693 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6333538e-9193-4692-946f-8f44c4869ceb-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-tvhls\" (UID: \"6333538e-9193-4692-946f-8f44c4869ceb\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-tvhls" Sep 08 08:10:43.153326 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:43.153309 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6333538e-9193-4692-946f-8f44c4869ceb-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-tvhls\" (UID: \"6333538e-9193-4692-946f-8f44c4869ceb\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-tvhls" Sep 08 08:10:43.160898 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:43.160879 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpswp\" (UniqueName: \"kubernetes.io/projected/6333538e-9193-4692-946f-8f44c4869ceb-kube-api-access-lpswp\") pod \"prometheus-operator-6f4946db56-tvhls\" (UID: \"6333538e-9193-4692-946f-8f44c4869ceb\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-tvhls" Sep 08 08:10:43.653746 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:43.653716 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/6333538e-9193-4692-946f-8f44c4869ceb-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-tvhls\" (UID: \"6333538e-9193-4692-946f-8f44c4869ceb\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-tvhls" Sep 08 08:10:43.655950 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:43.655930 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/6333538e-9193-4692-946f-8f44c4869ceb-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-tvhls\" (UID: \"6333538e-9193-4692-946f-8f44c4869ceb\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-tvhls" Sep 08 08:10:43.934729 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:43.934659 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-tvhls" Sep 08 08:10:44.207321 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:44.207254 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-tvhls"] Sep 08 08:10:44.209919 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:10:44.209894 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6333538e_9193_4692_946f_8f44c4869ceb.slice/crio-300744248dc859bea0956e681f5a6c210d81f5bd27f825ca65d182c239f5d198 WatchSource:0}: Error finding container 300744248dc859bea0956e681f5a6c210d81f5bd27f825ca65d182c239f5d198: Status 404 returned error can't find the container with id 300744248dc859bea0956e681f5a6c210d81f5bd27f825ca65d182c239f5d198 Sep 08 08:10:44.457933 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:44.457872 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-tvhls" event={"ID":"6333538e-9193-4692-946f-8f44c4869ceb","Type":"ContainerStarted","Data":"300744248dc859bea0956e681f5a6c210d81f5bd27f825ca65d182c239f5d198"} Sep 08 08:10:46.466038 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:46.466003 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-tvhls" event={"ID":"6333538e-9193-4692-946f-8f44c4869ceb","Type":"ContainerStarted","Data":"c1101a897a69b3262629d55be77eda180501c55c04c42b4a90bccfacb15483ee"} Sep 08 08:10:46.466414 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:46.466058 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-tvhls" event={"ID":"6333538e-9193-4692-946f-8f44c4869ceb","Type":"ContainerStarted","Data":"d38ffb19d8f7b2fb72690e452d6ab237cb2ecd1417902e65f653fd686786b262"} Sep 08 08:10:46.483487 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:46.483442 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-tvhls" podStartSLOduration=3.152267935 podStartE2EDuration="4.483426847s" totalImagesPullingTime="1.331158912s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 08:10:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:10:46.482973967 +0000 UTC m=+59.820613598" watchObservedRunningTime="2026-09-08 08:10:46.483426847 +0000 UTC m=+59.821066485" Sep 08 08:10:48.365471 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.365441 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-8qxcv"] Sep 08 08:10:48.384448 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.384428 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-8qxcv"] Sep 08 08:10:48.384591 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.384519 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-8qxcv" Sep 08 08:10:48.388093 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.388067 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 08 08:10:48.388093 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.388079 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 08 08:10:48.388336 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.388323 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-t795z\"" Sep 08 08:10:48.400141 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.399845 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-j5m5d"] Sep 08 08:10:48.412766 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.412748 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-j5m5d" Sep 08 08:10:48.415745 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.415723 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 08 08:10:48.415855 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.415792 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 08 08:10:48.415987 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.415972 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 08 08:10:48.416680 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.416660 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-ssksm\"" Sep 08 08:10:48.484292 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.484270 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rk9hw\" (UniqueName: \"kubernetes.io/projected/ecca6e11-9831-4fe4-9e0d-aacadb1a404c-kube-api-access-rk9hw\") pod \"node-exporter-j5m5d\" (UID: \"ecca6e11-9831-4fe4-9e0d-aacadb1a404c\") " pod="openshift-monitoring/node-exporter-j5m5d" Sep 08 08:10:48.484430 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.484311 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ecca6e11-9831-4fe4-9e0d-aacadb1a404c-node-exporter-textfile\") pod \"node-exporter-j5m5d\" (UID: \"ecca6e11-9831-4fe4-9e0d-aacadb1a404c\") " pod="openshift-monitoring/node-exporter-j5m5d" Sep 08 08:10:48.484430 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.484333 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ecca6e11-9831-4fe4-9e0d-aacadb1a404c-metrics-client-ca\") pod \"node-exporter-j5m5d\" (UID: \"ecca6e11-9831-4fe4-9e0d-aacadb1a404c\") " pod="openshift-monitoring/node-exporter-j5m5d" Sep 08 08:10:48.484430 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.484374 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45b90f7c-de16-4a7f-b5ab-9cd28caa701c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-8qxcv\" (UID: \"45b90f7c-de16-4a7f-b5ab-9cd28caa701c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-8qxcv" Sep 08 08:10:48.484542 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.484441 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ecca6e11-9831-4fe4-9e0d-aacadb1a404c-node-exporter-tls\") pod \"node-exporter-j5m5d\" (UID: \"ecca6e11-9831-4fe4-9e0d-aacadb1a404c\") " pod="openshift-monitoring/node-exporter-j5m5d" Sep 08 08:10:48.484542 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.484458 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ecca6e11-9831-4fe4-9e0d-aacadb1a404c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-j5m5d\" (UID: \"ecca6e11-9831-4fe4-9e0d-aacadb1a404c\") " pod="openshift-monitoring/node-exporter-j5m5d" Sep 08 08:10:48.484542 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.484477 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ecca6e11-9831-4fe4-9e0d-aacadb1a404c-sys\") pod \"node-exporter-j5m5d\" (UID: \"ecca6e11-9831-4fe4-9e0d-aacadb1a404c\") " pod="openshift-monitoring/node-exporter-j5m5d" Sep 08 08:10:48.484542 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.484497 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45b90f7c-de16-4a7f-b5ab-9cd28caa701c-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-8qxcv\" (UID: \"45b90f7c-de16-4a7f-b5ab-9cd28caa701c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-8qxcv" Sep 08 08:10:48.484542 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.484520 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ecca6e11-9831-4fe4-9e0d-aacadb1a404c-node-exporter-wtmp\") pod \"node-exporter-j5m5d\" (UID: \"ecca6e11-9831-4fe4-9e0d-aacadb1a404c\") " pod="openshift-monitoring/node-exporter-j5m5d" Sep 08 08:10:48.484542 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.484535 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ecca6e11-9831-4fe4-9e0d-aacadb1a404c-root\") pod \"node-exporter-j5m5d\" (UID: \"ecca6e11-9831-4fe4-9e0d-aacadb1a404c\") " pod="openshift-monitoring/node-exporter-j5m5d" Sep 08 08:10:48.484747 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.484592 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/45b90f7c-de16-4a7f-b5ab-9cd28caa701c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-8qxcv\" (UID: \"45b90f7c-de16-4a7f-b5ab-9cd28caa701c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-8qxcv" Sep 08 08:10:48.484747 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.484607 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nscqg\" (UniqueName: \"kubernetes.io/projected/45b90f7c-de16-4a7f-b5ab-9cd28caa701c-kube-api-access-nscqg\") pod \"openshift-state-metrics-657fd6c54-8qxcv\" (UID: \"45b90f7c-de16-4a7f-b5ab-9cd28caa701c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-8qxcv" Sep 08 08:10:48.484747 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.484625 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ecca6e11-9831-4fe4-9e0d-aacadb1a404c-node-exporter-accelerators-collector-config\") pod \"node-exporter-j5m5d\" (UID: \"ecca6e11-9831-4fe4-9e0d-aacadb1a404c\") " pod="openshift-monitoring/node-exporter-j5m5d" Sep 08 08:10:48.585532 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.585507 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ecca6e11-9831-4fe4-9e0d-aacadb1a404c-metrics-client-ca\") pod \"node-exporter-j5m5d\" (UID: \"ecca6e11-9831-4fe4-9e0d-aacadb1a404c\") " pod="openshift-monitoring/node-exporter-j5m5d" Sep 08 08:10:48.585648 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.585536 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45b90f7c-de16-4a7f-b5ab-9cd28caa701c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-8qxcv\" (UID: \"45b90f7c-de16-4a7f-b5ab-9cd28caa701c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-8qxcv" Sep 08 08:10:48.585648 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.585563 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ecca6e11-9831-4fe4-9e0d-aacadb1a404c-node-exporter-tls\") pod \"node-exporter-j5m5d\" (UID: \"ecca6e11-9831-4fe4-9e0d-aacadb1a404c\") " pod="openshift-monitoring/node-exporter-j5m5d" Sep 08 08:10:48.585648 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.585579 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ecca6e11-9831-4fe4-9e0d-aacadb1a404c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-j5m5d\" (UID: \"ecca6e11-9831-4fe4-9e0d-aacadb1a404c\") " pod="openshift-monitoring/node-exporter-j5m5d" Sep 08 08:10:48.585648 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.585597 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ecca6e11-9831-4fe4-9e0d-aacadb1a404c-sys\") pod \"node-exporter-j5m5d\" (UID: \"ecca6e11-9831-4fe4-9e0d-aacadb1a404c\") " pod="openshift-monitoring/node-exporter-j5m5d" Sep 08 08:10:48.585648 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.585612 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45b90f7c-de16-4a7f-b5ab-9cd28caa701c-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-8qxcv\" (UID: \"45b90f7c-de16-4a7f-b5ab-9cd28caa701c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-8qxcv" Sep 08 08:10:48.585648 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.585646 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ecca6e11-9831-4fe4-9e0d-aacadb1a404c-node-exporter-wtmp\") pod \"node-exporter-j5m5d\" (UID: \"ecca6e11-9831-4fe4-9e0d-aacadb1a404c\") " pod="openshift-monitoring/node-exporter-j5m5d" Sep 08 08:10:48.585947 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.585671 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ecca6e11-9831-4fe4-9e0d-aacadb1a404c-root\") pod \"node-exporter-j5m5d\" (UID: \"ecca6e11-9831-4fe4-9e0d-aacadb1a404c\") " pod="openshift-monitoring/node-exporter-j5m5d" Sep 08 08:10:48.585947 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:48.585679 2585 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 08 08:10:48.585947 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.585686 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ecca6e11-9831-4fe4-9e0d-aacadb1a404c-sys\") pod \"node-exporter-j5m5d\" (UID: \"ecca6e11-9831-4fe4-9e0d-aacadb1a404c\") " pod="openshift-monitoring/node-exporter-j5m5d" Sep 08 08:10:48.585947 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.585736 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ecca6e11-9831-4fe4-9e0d-aacadb1a404c-root\") pod \"node-exporter-j5m5d\" (UID: \"ecca6e11-9831-4fe4-9e0d-aacadb1a404c\") " pod="openshift-monitoring/node-exporter-j5m5d" Sep 08 08:10:48.585947 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:48.585741 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecca6e11-9831-4fe4-9e0d-aacadb1a404c-node-exporter-tls podName:ecca6e11-9831-4fe4-9e0d-aacadb1a404c nodeName:}" failed. No retries permitted until 2026-09-08 08:10:49.085721779 +0000 UTC m=+62.423361400 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/ecca6e11-9831-4fe4-9e0d-aacadb1a404c-node-exporter-tls") pod "node-exporter-j5m5d" (UID: "ecca6e11-9831-4fe4-9e0d-aacadb1a404c") : secret "node-exporter-tls" not found Sep 08 08:10:48.585947 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.585765 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/45b90f7c-de16-4a7f-b5ab-9cd28caa701c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-8qxcv\" (UID: \"45b90f7c-de16-4a7f-b5ab-9cd28caa701c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-8qxcv" Sep 08 08:10:48.585947 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.585802 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nscqg\" (UniqueName: \"kubernetes.io/projected/45b90f7c-de16-4a7f-b5ab-9cd28caa701c-kube-api-access-nscqg\") pod \"openshift-state-metrics-657fd6c54-8qxcv\" (UID: \"45b90f7c-de16-4a7f-b5ab-9cd28caa701c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-8qxcv" Sep 08 08:10:48.585947 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.585835 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ecca6e11-9831-4fe4-9e0d-aacadb1a404c-node-exporter-accelerators-collector-config\") pod \"node-exporter-j5m5d\" (UID: \"ecca6e11-9831-4fe4-9e0d-aacadb1a404c\") " pod="openshift-monitoring/node-exporter-j5m5d" Sep 08 08:10:48.585947 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.585853 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ecca6e11-9831-4fe4-9e0d-aacadb1a404c-node-exporter-wtmp\") pod \"node-exporter-j5m5d\" (UID: \"ecca6e11-9831-4fe4-9e0d-aacadb1a404c\") " pod="openshift-monitoring/node-exporter-j5m5d" Sep 08 08:10:48.585947 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.585859 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk9hw\" (UniqueName: \"kubernetes.io/projected/ecca6e11-9831-4fe4-9e0d-aacadb1a404c-kube-api-access-rk9hw\") pod \"node-exporter-j5m5d\" (UID: \"ecca6e11-9831-4fe4-9e0d-aacadb1a404c\") " pod="openshift-monitoring/node-exporter-j5m5d" Sep 08 08:10:48.585947 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.585924 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ecca6e11-9831-4fe4-9e0d-aacadb1a404c-node-exporter-textfile\") pod \"node-exporter-j5m5d\" (UID: \"ecca6e11-9831-4fe4-9e0d-aacadb1a404c\") " pod="openshift-monitoring/node-exporter-j5m5d" Sep 08 08:10:48.586674 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.586115 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ecca6e11-9831-4fe4-9e0d-aacadb1a404c-metrics-client-ca\") pod \"node-exporter-j5m5d\" (UID: \"ecca6e11-9831-4fe4-9e0d-aacadb1a404c\") " pod="openshift-monitoring/node-exporter-j5m5d" Sep 08 08:10:48.586674 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.586183 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ecca6e11-9831-4fe4-9e0d-aacadb1a404c-node-exporter-textfile\") pod \"node-exporter-j5m5d\" (UID: \"ecca6e11-9831-4fe4-9e0d-aacadb1a404c\") " pod="openshift-monitoring/node-exporter-j5m5d" Sep 08 08:10:48.586674 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.586450 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45b90f7c-de16-4a7f-b5ab-9cd28caa701c-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-8qxcv\" (UID: \"45b90f7c-de16-4a7f-b5ab-9cd28caa701c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-8qxcv" Sep 08 08:10:48.586826 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.586801 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ecca6e11-9831-4fe4-9e0d-aacadb1a404c-node-exporter-accelerators-collector-config\") pod \"node-exporter-j5m5d\" (UID: \"ecca6e11-9831-4fe4-9e0d-aacadb1a404c\") " pod="openshift-monitoring/node-exporter-j5m5d" Sep 08 08:10:48.588134 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.588116 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ecca6e11-9831-4fe4-9e0d-aacadb1a404c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-j5m5d\" (UID: \"ecca6e11-9831-4fe4-9e0d-aacadb1a404c\") " pod="openshift-monitoring/node-exporter-j5m5d" Sep 08 08:10:48.588219 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.588190 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45b90f7c-de16-4a7f-b5ab-9cd28caa701c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-8qxcv\" (UID: \"45b90f7c-de16-4a7f-b5ab-9cd28caa701c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-8qxcv" Sep 08 08:10:48.588405 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.588367 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/45b90f7c-de16-4a7f-b5ab-9cd28caa701c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-8qxcv\" (UID: \"45b90f7c-de16-4a7f-b5ab-9cd28caa701c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-8qxcv" Sep 08 08:10:48.598005 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.597982 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rk9hw\" (UniqueName: \"kubernetes.io/projected/ecca6e11-9831-4fe4-9e0d-aacadb1a404c-kube-api-access-rk9hw\") pod \"node-exporter-j5m5d\" (UID: \"ecca6e11-9831-4fe4-9e0d-aacadb1a404c\") " pod="openshift-monitoring/node-exporter-j5m5d" Sep 08 08:10:48.598097 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.598040 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nscqg\" (UniqueName: \"kubernetes.io/projected/45b90f7c-de16-4a7f-b5ab-9cd28caa701c-kube-api-access-nscqg\") pod \"openshift-state-metrics-657fd6c54-8qxcv\" (UID: \"45b90f7c-de16-4a7f-b5ab-9cd28caa701c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-8qxcv" Sep 08 08:10:48.692685 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.692664 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-8qxcv" Sep 08 08:10:48.974179 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:48.974114 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-8qxcv"] Sep 08 08:10:48.977173 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:10:48.977147 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45b90f7c_de16_4a7f_b5ab_9cd28caa701c.slice/crio-7471a8cb9b9070fcdab7301ff671a31dd0624e8d9c20842d93d80aafdd04785f WatchSource:0}: Error finding container 7471a8cb9b9070fcdab7301ff671a31dd0624e8d9c20842d93d80aafdd04785f: Status 404 returned error can't find the container with id 7471a8cb9b9070fcdab7301ff671a31dd0624e8d9c20842d93d80aafdd04785f Sep 08 08:10:49.089429 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.089402 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ecca6e11-9831-4fe4-9e0d-aacadb1a404c-node-exporter-tls\") pod \"node-exporter-j5m5d\" (UID: \"ecca6e11-9831-4fe4-9e0d-aacadb1a404c\") " pod="openshift-monitoring/node-exporter-j5m5d" Sep 08 08:10:49.091686 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.091667 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ecca6e11-9831-4fe4-9e0d-aacadb1a404c-node-exporter-tls\") pod \"node-exporter-j5m5d\" (UID: \"ecca6e11-9831-4fe4-9e0d-aacadb1a404c\") " pod="openshift-monitoring/node-exporter-j5m5d" Sep 08 08:10:49.320947 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.320883 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-j5m5d" Sep 08 08:10:49.362146 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:10:49.362117 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecca6e11_9831_4fe4_9e0d_aacadb1a404c.slice/crio-30fbc32535a271e42bb894f5c45575ef9b0f6fbdfb45e10bf94e3b85eccfe401 WatchSource:0}: Error finding container 30fbc32535a271e42bb894f5c45575ef9b0f6fbdfb45e10bf94e3b85eccfe401: Status 404 returned error can't find the container with id 30fbc32535a271e42bb894f5c45575ef9b0f6fbdfb45e10bf94e3b85eccfe401 Sep 08 08:10:49.389137 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.389117 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2xlvk" Sep 08 08:10:49.445903 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.445876 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-rdr6d" Sep 08 08:10:49.452476 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.452453 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 08:10:49.473652 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.473627 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:49.474685 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.474658 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 08:10:49.477137 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.477114 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 08 08:10:49.477251 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.477170 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 08 08:10:49.477615 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.477371 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 08 08:10:49.477737 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.477638 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 08 08:10:49.478089 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.477891 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 08 08:10:49.478089 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.477906 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 08 08:10:49.478089 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.477937 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 08 08:10:49.478089 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.477942 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 08 08:10:49.478089 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.477900 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-cz9hw\"" Sep 08 08:10:49.478089 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.477892 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 08 08:10:49.478466 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.478163 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-j5m5d" event={"ID":"ecca6e11-9831-4fe4-9e0d-aacadb1a404c","Type":"ContainerStarted","Data":"30fbc32535a271e42bb894f5c45575ef9b0f6fbdfb45e10bf94e3b85eccfe401"} Sep 08 08:10:49.480812 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.480789 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-8qxcv" event={"ID":"45b90f7c-de16-4a7f-b5ab-9cd28caa701c","Type":"ContainerStarted","Data":"08dffd4180a1ee8a432c6e2f6adaad0e02c7762a9638fa46254afec739656070"} Sep 08 08:10:49.480950 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.480851 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-8qxcv" event={"ID":"45b90f7c-de16-4a7f-b5ab-9cd28caa701c","Type":"ContainerStarted","Data":"8b9ce991b0b72a62800ee4feed83b2b58328b8dba2fe383fe0c44f38a562aaae"} Sep 08 08:10:49.480950 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.480882 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-8qxcv" event={"ID":"45b90f7c-de16-4a7f-b5ab-9cd28caa701c","Type":"ContainerStarted","Data":"7471a8cb9b9070fcdab7301ff671a31dd0624e8d9c20842d93d80aafdd04785f"} Sep 08 08:10:49.492885 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.492861 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7932f481-a24e-4554-9401-b69cf440085e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:49.492985 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.492894 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plhxr\" (UniqueName: \"kubernetes.io/projected/7932f481-a24e-4554-9401-b69cf440085e-kube-api-access-plhxr\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:49.492985 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.492932 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7932f481-a24e-4554-9401-b69cf440085e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:49.493092 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.493066 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7932f481-a24e-4554-9401-b69cf440085e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:49.493147 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.493092 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7932f481-a24e-4554-9401-b69cf440085e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:49.493196 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.493153 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7932f481-a24e-4554-9401-b69cf440085e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:49.493246 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.493217 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7932f481-a24e-4554-9401-b69cf440085e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:49.493293 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.493261 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7932f481-a24e-4554-9401-b69cf440085e-config-out\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:49.493844 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.493771 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7932f481-a24e-4554-9401-b69cf440085e-config-volume\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:49.493844 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.493802 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7932f481-a24e-4554-9401-b69cf440085e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:49.494242 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.494153 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7932f481-a24e-4554-9401-b69cf440085e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:49.494242 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.494204 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7932f481-a24e-4554-9401-b69cf440085e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:49.494242 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.494226 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7932f481-a24e-4554-9401-b69cf440085e-web-config\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:49.595130 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.595052 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7932f481-a24e-4554-9401-b69cf440085e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:49.595130 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.595093 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7932f481-a24e-4554-9401-b69cf440085e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:49.595130 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.595115 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7932f481-a24e-4554-9401-b69cf440085e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:49.595362 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.595313 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7932f481-a24e-4554-9401-b69cf440085e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:49.595362 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.595352 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7932f481-a24e-4554-9401-b69cf440085e-config-out\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:49.595495 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.595395 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7932f481-a24e-4554-9401-b69cf440085e-config-volume\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:49.595495 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.595423 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7932f481-a24e-4554-9401-b69cf440085e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:49.595495 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.595477 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7932f481-a24e-4554-9401-b69cf440085e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:49.595640 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.595516 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7932f481-a24e-4554-9401-b69cf440085e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:49.595640 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.595549 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7932f481-a24e-4554-9401-b69cf440085e-web-config\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:49.595640 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.595606 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7932f481-a24e-4554-9401-b69cf440085e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:49.595795 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.595642 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plhxr\" (UniqueName: \"kubernetes.io/projected/7932f481-a24e-4554-9401-b69cf440085e-kube-api-access-plhxr\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:49.595795 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.595672 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7932f481-a24e-4554-9401-b69cf440085e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:49.595795 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:49.595788 2585 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Sep 08 08:10:49.596223 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:49.596115 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7932f481-a24e-4554-9401-b69cf440085e-secret-alertmanager-main-tls podName:7932f481-a24e-4554-9401-b69cf440085e nodeName:}" failed. No retries permitted until 2026-09-08 08:10:50.095827389 +0000 UTC m=+63.433467023 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/7932f481-a24e-4554-9401-b69cf440085e-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "7932f481-a24e-4554-9401-b69cf440085e") : secret "alertmanager-main-tls" not found Sep 08 08:10:49.596420 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:10:49.596298 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7932f481-a24e-4554-9401-b69cf440085e-alertmanager-trusted-ca-bundle podName:7932f481-a24e-4554-9401-b69cf440085e nodeName:}" failed. No retries permitted until 2026-09-08 08:10:50.096263253 +0000 UTC m=+63.433902874 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/7932f481-a24e-4554-9401-b69cf440085e-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "7932f481-a24e-4554-9401-b69cf440085e") : configmap references non-existent config key: ca-bundle.crt Sep 08 08:10:49.605229 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.605182 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7932f481-a24e-4554-9401-b69cf440085e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:49.605325 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.605295 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7932f481-a24e-4554-9401-b69cf440085e-config-out\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:49.605670 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.605461 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7932f481-a24e-4554-9401-b69cf440085e-config-volume\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:49.605670 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.605463 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7932f481-a24e-4554-9401-b69cf440085e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:49.605670 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.605535 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7932f481-a24e-4554-9401-b69cf440085e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:49.605670 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.605641 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7932f481-a24e-4554-9401-b69cf440085e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:49.605670 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.605649 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7932f481-a24e-4554-9401-b69cf440085e-web-config\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:49.606060 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.605752 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7932f481-a24e-4554-9401-b69cf440085e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:49.606060 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.605970 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7932f481-a24e-4554-9401-b69cf440085e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:49.606466 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.606233 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7932f481-a24e-4554-9401-b69cf440085e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:49.608471 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:49.608447 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plhxr\" (UniqueName: \"kubernetes.io/projected/7932f481-a24e-4554-9401-b69cf440085e-kube-api-access-plhxr\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:50.098611 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:50.098578 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7932f481-a24e-4554-9401-b69cf440085e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:50.098611 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:50.098618 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7932f481-a24e-4554-9401-b69cf440085e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:50.099502 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:50.099460 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7932f481-a24e-4554-9401-b69cf440085e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:50.101266 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:50.101239 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7932f481-a24e-4554-9401-b69cf440085e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7932f481-a24e-4554-9401-b69cf440085e\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:50.384646 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:50.384603 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:10:50.487135 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:50.486997 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-8qxcv" event={"ID":"45b90f7c-de16-4a7f-b5ab-9cd28caa701c","Type":"ContainerStarted","Data":"2eb1e959d223b5ca63287ecfe3de9eb94d85f7a50e6137e15415befaebfd1320"} Sep 08 08:10:50.507164 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:50.507120 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-8qxcv" podStartSLOduration=1.345278677 podStartE2EDuration="2.507104191s" totalImagesPullingTime="1.161825514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 08:10:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:10:50.505775455 +0000 UTC m=+63.843415096" watchObservedRunningTime="2026-09-08 08:10:50.507104191 +0000 UTC m=+63.844743831" Sep 08 08:10:50.676686 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:50.676599 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 08:10:50.691368 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:10:50.691339 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7932f481_a24e_4554_9401_b69cf440085e.slice/crio-73dfdfba8ce452b3e9f208fd564b7ea3d4e7696e64a8ab82cff922f26eed7099 WatchSource:0}: Error finding container 73dfdfba8ce452b3e9f208fd564b7ea3d4e7696e64a8ab82cff922f26eed7099: Status 404 returned error can't find the container with id 73dfdfba8ce452b3e9f208fd564b7ea3d4e7696e64a8ab82cff922f26eed7099 Sep 08 08:10:51.489688 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:51.489655 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7699cb4d56-j2wp8"] Sep 08 08:10:51.492020 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:51.491991 2585 generic.go:356] "Generic (PLEG): container finished" podID="ecca6e11-9831-4fe4-9e0d-aacadb1a404c" containerID="863d07398a0f9bb83d9274cfee1e8c579f6699ebbd5ac50e41b27e1e005dd576" exitCode=0 Sep 08 08:10:51.492159 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:51.492023 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7699cb4d56-j2wp8" Sep 08 08:10:51.492159 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:51.492024 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-j5m5d" event={"ID":"ecca6e11-9831-4fe4-9e0d-aacadb1a404c","Type":"ContainerDied","Data":"863d07398a0f9bb83d9274cfee1e8c579f6699ebbd5ac50e41b27e1e005dd576"} Sep 08 08:10:51.494189 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:51.494167 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7932f481-a24e-4554-9401-b69cf440085e","Type":"ContainerStarted","Data":"73dfdfba8ce452b3e9f208fd564b7ea3d4e7696e64a8ab82cff922f26eed7099"} Sep 08 08:10:51.498580 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:51.498558 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Sep 08 08:10:51.499081 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:51.498664 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-ba84a5tkepf4d\"" Sep 08 08:10:51.499081 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:51.498713 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Sep 08 08:10:51.499081 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:51.498773 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Sep 08 08:10:51.499081 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:51.498894 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Sep 08 08:10:51.499081 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:51.499002 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-q9t6c\"" Sep 08 08:10:51.499081 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:51.499074 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Sep 08 08:10:51.507717 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:51.507654 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7699cb4d56-j2wp8"] Sep 08 08:10:51.508994 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:51.508970 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/edebb973-5c2f-4951-971b-eb30724528c9-secret-thanos-querier-tls\") pod \"thanos-querier-7699cb4d56-j2wp8\" (UID: \"edebb973-5c2f-4951-971b-eb30724528c9\") " pod="openshift-monitoring/thanos-querier-7699cb4d56-j2wp8" Sep 08 08:10:51.509087 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:51.509015 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/edebb973-5c2f-4951-971b-eb30724528c9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7699cb4d56-j2wp8\" (UID: \"edebb973-5c2f-4951-971b-eb30724528c9\") " pod="openshift-monitoring/thanos-querier-7699cb4d56-j2wp8" Sep 08 08:10:51.509150 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:51.509107 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/edebb973-5c2f-4951-971b-eb30724528c9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7699cb4d56-j2wp8\" (UID: \"edebb973-5c2f-4951-971b-eb30724528c9\") " pod="openshift-monitoring/thanos-querier-7699cb4d56-j2wp8" Sep 08 08:10:51.509205 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:51.509159 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/edebb973-5c2f-4951-971b-eb30724528c9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7699cb4d56-j2wp8\" (UID: \"edebb973-5c2f-4951-971b-eb30724528c9\") " pod="openshift-monitoring/thanos-querier-7699cb4d56-j2wp8" Sep 08 08:10:51.509205 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:51.509188 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/edebb973-5c2f-4951-971b-eb30724528c9-metrics-client-ca\") pod \"thanos-querier-7699cb4d56-j2wp8\" (UID: \"edebb973-5c2f-4951-971b-eb30724528c9\") " pod="openshift-monitoring/thanos-querier-7699cb4d56-j2wp8" Sep 08 08:10:51.509315 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:51.509214 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghwgn\" (UniqueName: \"kubernetes.io/projected/edebb973-5c2f-4951-971b-eb30724528c9-kube-api-access-ghwgn\") pod \"thanos-querier-7699cb4d56-j2wp8\" (UID: \"edebb973-5c2f-4951-971b-eb30724528c9\") " pod="openshift-monitoring/thanos-querier-7699cb4d56-j2wp8" Sep 08 08:10:51.509467 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:51.509432 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/edebb973-5c2f-4951-971b-eb30724528c9-secret-grpc-tls\") pod \"thanos-querier-7699cb4d56-j2wp8\" (UID: \"edebb973-5c2f-4951-971b-eb30724528c9\") " pod="openshift-monitoring/thanos-querier-7699cb4d56-j2wp8" Sep 08 08:10:51.509573 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:51.509487 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/edebb973-5c2f-4951-971b-eb30724528c9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7699cb4d56-j2wp8\" (UID: \"edebb973-5c2f-4951-971b-eb30724528c9\") " pod="openshift-monitoring/thanos-querier-7699cb4d56-j2wp8" Sep 08 08:10:51.610675 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:51.610649 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/edebb973-5c2f-4951-971b-eb30724528c9-secret-grpc-tls\") pod \"thanos-querier-7699cb4d56-j2wp8\" (UID: \"edebb973-5c2f-4951-971b-eb30724528c9\") " pod="openshift-monitoring/thanos-querier-7699cb4d56-j2wp8" Sep 08 08:10:51.610789 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:51.610684 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/edebb973-5c2f-4951-971b-eb30724528c9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7699cb4d56-j2wp8\" (UID: \"edebb973-5c2f-4951-971b-eb30724528c9\") " pod="openshift-monitoring/thanos-querier-7699cb4d56-j2wp8" Sep 08 08:10:51.610789 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:51.610731 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/edebb973-5c2f-4951-971b-eb30724528c9-secret-thanos-querier-tls\") pod \"thanos-querier-7699cb4d56-j2wp8\" (UID: \"edebb973-5c2f-4951-971b-eb30724528c9\") " pod="openshift-monitoring/thanos-querier-7699cb4d56-j2wp8" Sep 08 08:10:51.610789 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:51.610761 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/edebb973-5c2f-4951-971b-eb30724528c9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7699cb4d56-j2wp8\" (UID: \"edebb973-5c2f-4951-971b-eb30724528c9\") " pod="openshift-monitoring/thanos-querier-7699cb4d56-j2wp8" Sep 08 08:10:51.610789 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:51.610780 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/edebb973-5c2f-4951-971b-eb30724528c9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7699cb4d56-j2wp8\" (UID: \"edebb973-5c2f-4951-971b-eb30724528c9\") " pod="openshift-monitoring/thanos-querier-7699cb4d56-j2wp8" Sep 08 08:10:51.610996 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:51.610812 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/edebb973-5c2f-4951-971b-eb30724528c9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7699cb4d56-j2wp8\" (UID: \"edebb973-5c2f-4951-971b-eb30724528c9\") " pod="openshift-monitoring/thanos-querier-7699cb4d56-j2wp8" Sep 08 08:10:51.610996 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:51.610835 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/edebb973-5c2f-4951-971b-eb30724528c9-metrics-client-ca\") pod \"thanos-querier-7699cb4d56-j2wp8\" (UID: \"edebb973-5c2f-4951-971b-eb30724528c9\") " pod="openshift-monitoring/thanos-querier-7699cb4d56-j2wp8" Sep 08 08:10:51.610996 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:51.610938 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghwgn\" (UniqueName: \"kubernetes.io/projected/edebb973-5c2f-4951-971b-eb30724528c9-kube-api-access-ghwgn\") pod \"thanos-querier-7699cb4d56-j2wp8\" (UID: \"edebb973-5c2f-4951-971b-eb30724528c9\") " pod="openshift-monitoring/thanos-querier-7699cb4d56-j2wp8" Sep 08 08:10:51.611642 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:51.611595 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/edebb973-5c2f-4951-971b-eb30724528c9-metrics-client-ca\") pod \"thanos-querier-7699cb4d56-j2wp8\" (UID: \"edebb973-5c2f-4951-971b-eb30724528c9\") " pod="openshift-monitoring/thanos-querier-7699cb4d56-j2wp8" Sep 08 08:10:51.613888 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:51.613863 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/edebb973-5c2f-4951-971b-eb30724528c9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7699cb4d56-j2wp8\" (UID: \"edebb973-5c2f-4951-971b-eb30724528c9\") " pod="openshift-monitoring/thanos-querier-7699cb4d56-j2wp8" Sep 08 08:10:51.613984 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:51.613927 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/edebb973-5c2f-4951-971b-eb30724528c9-secret-grpc-tls\") pod \"thanos-querier-7699cb4d56-j2wp8\" (UID: \"edebb973-5c2f-4951-971b-eb30724528c9\") " pod="openshift-monitoring/thanos-querier-7699cb4d56-j2wp8" Sep 08 08:10:51.613984 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:51.613940 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/edebb973-5c2f-4951-971b-eb30724528c9-secret-thanos-querier-tls\") pod \"thanos-querier-7699cb4d56-j2wp8\" (UID: \"edebb973-5c2f-4951-971b-eb30724528c9\") " pod="openshift-monitoring/thanos-querier-7699cb4d56-j2wp8" Sep 08 08:10:51.613984 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:51.613965 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/edebb973-5c2f-4951-971b-eb30724528c9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7699cb4d56-j2wp8\" (UID: \"edebb973-5c2f-4951-971b-eb30724528c9\") " pod="openshift-monitoring/thanos-querier-7699cb4d56-j2wp8" Sep 08 08:10:51.614435 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:51.614409 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/edebb973-5c2f-4951-971b-eb30724528c9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7699cb4d56-j2wp8\" (UID: \"edebb973-5c2f-4951-971b-eb30724528c9\") " pod="openshift-monitoring/thanos-querier-7699cb4d56-j2wp8" Sep 08 08:10:51.614569 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:51.614472 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/edebb973-5c2f-4951-971b-eb30724528c9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7699cb4d56-j2wp8\" (UID: \"edebb973-5c2f-4951-971b-eb30724528c9\") " pod="openshift-monitoring/thanos-querier-7699cb4d56-j2wp8" Sep 08 08:10:51.618345 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:51.618318 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghwgn\" (UniqueName: \"kubernetes.io/projected/edebb973-5c2f-4951-971b-eb30724528c9-kube-api-access-ghwgn\") pod \"thanos-querier-7699cb4d56-j2wp8\" (UID: \"edebb973-5c2f-4951-971b-eb30724528c9\") " pod="openshift-monitoring/thanos-querier-7699cb4d56-j2wp8" Sep 08 08:10:51.802582 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:51.802519 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7699cb4d56-j2wp8" Sep 08 08:10:52.082794 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:52.082715 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7699cb4d56-j2wp8"] Sep 08 08:10:52.085650 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:10:52.085625 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedebb973_5c2f_4951_971b_eb30724528c9.slice/crio-4aaa66002c4b9bf7bc5e595b88a85c31daf66c700a08b92b3f72a310c55ab466 WatchSource:0}: Error finding container 4aaa66002c4b9bf7bc5e595b88a85c31daf66c700a08b92b3f72a310c55ab466: Status 404 returned error can't find the container with id 4aaa66002c4b9bf7bc5e595b88a85c31daf66c700a08b92b3f72a310c55ab466 Sep 08 08:10:52.498833 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:52.498801 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7699cb4d56-j2wp8" event={"ID":"edebb973-5c2f-4951-971b-eb30724528c9","Type":"ContainerStarted","Data":"4aaa66002c4b9bf7bc5e595b88a85c31daf66c700a08b92b3f72a310c55ab466"} Sep 08 08:10:52.501256 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:52.501228 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-j5m5d" event={"ID":"ecca6e11-9831-4fe4-9e0d-aacadb1a404c","Type":"ContainerStarted","Data":"7b69ebc5792613c54a37acf5feda6c392a4c6fc07ef258e49756b238406351bc"} Sep 08 08:10:52.501410 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:52.501275 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-j5m5d" event={"ID":"ecca6e11-9831-4fe4-9e0d-aacadb1a404c","Type":"ContainerStarted","Data":"34bce53ed3e91c7ba27f447a4bcb43fb324d304c9b8f8198eb9e68b9ba233d84"} Sep 08 08:10:52.522504 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:52.522456 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-j5m5d" podStartSLOduration=3.160464958 podStartE2EDuration="4.52243591s" totalImagesPullingTime="1.294236935s" totalInitContainerRuntime="67.734017ms" isStatefulPod=true podCreationTimestamp="2026-09-08 08:10:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:10:52.520726688 +0000 UTC m=+65.858366330" watchObservedRunningTime="2026-09-08 08:10:52.52243591 +0000 UTC m=+65.860075552" Sep 08 08:10:52.820049 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:52.820018 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95940b17-787c-4892-96f0-83cf9501e4d5-metrics-certs\") pod \"network-metrics-daemon-ns5tq\" (UID: \"95940b17-787c-4892-96f0-83cf9501e4d5\") " pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:10:52.822344 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:52.822319 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 08 08:10:52.834124 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:52.834098 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95940b17-787c-4892-96f0-83cf9501e4d5-metrics-certs\") pod \"network-metrics-daemon-ns5tq\" (UID: \"95940b17-787c-4892-96f0-83cf9501e4d5\") " pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:10:52.920892 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:52.920858 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dac5eaa3-6c21-4b05-ace5-a13c8344ab6e-original-pull-secret\") pod \"global-pull-secret-syncer-mczjr\" (UID: \"dac5eaa3-6c21-4b05-ace5-a13c8344ab6e\") " pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:10:52.923861 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:52.923840 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 08 08:10:52.934421 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:52.934400 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dac5eaa3-6c21-4b05-ace5-a13c8344ab6e-original-pull-secret\") pod \"global-pull-secret-syncer-mczjr\" (UID: \"dac5eaa3-6c21-4b05-ace5-a13c8344ab6e\") " pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:10:52.980530 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:52.980507 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-n4fwr\"" Sep 08 08:10:52.989096 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:52.989076 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns5tq" Sep 08 08:10:53.021546 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:53.021471 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlnps\" (UniqueName: \"kubernetes.io/projected/7301238b-b7d9-4101-8f7a-5e442499ca05-kube-api-access-jlnps\") pod \"network-check-target-k45rp\" (UID: \"7301238b-b7d9-4101-8f7a-5e442499ca05\") " pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:10:53.024184 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:53.024161 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 08 08:10:53.034830 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:53.034809 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 08 08:10:53.067421 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:53.066800 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlnps\" (UniqueName: \"kubernetes.io/projected/7301238b-b7d9-4101-8f7a-5e442499ca05-kube-api-access-jlnps\") pod \"network-check-target-k45rp\" (UID: \"7301238b-b7d9-4101-8f7a-5e442499ca05\") " pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:10:53.084425 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:53.084394 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mczjr" Sep 08 08:10:53.091912 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:53.091893 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vr4wt\"" Sep 08 08:10:53.099942 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:53.099859 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:10:53.181289 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:53.180766 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-5ncz8"] Sep 08 08:10:53.187492 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:53.187456 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-5ncz8" Sep 08 08:10:53.190073 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:53.190051 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Sep 08 08:10:53.190447 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:53.190430 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-knf2l\"" Sep 08 08:10:53.196760 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:53.196706 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-5ncz8"] Sep 08 08:10:53.223295 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:53.223271 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/69650628-eaa4-400d-9c77-eacece44749a-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-5ncz8\" (UID: \"69650628-eaa4-400d-9c77-eacece44749a\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-5ncz8" Sep 08 08:10:53.223428 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:53.223327 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djvz2\" (UniqueName: \"kubernetes.io/projected/69650628-eaa4-400d-9c77-eacece44749a-kube-api-access-djvz2\") pod \"monitoring-plugin-85bbbbbf4-5ncz8\" (UID: \"69650628-eaa4-400d-9c77-eacece44749a\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-5ncz8" Sep 08 08:10:53.274812 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:53.274748 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ns5tq"] Sep 08 08:10:53.278470 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:10:53.278448 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95940b17_787c_4892_96f0_83cf9501e4d5.slice/crio-cba2b2889e05e1dcb9329affe90c0f6fca108619e44afb4cdf30ec0fb24fcec6 WatchSource:0}: Error finding container cba2b2889e05e1dcb9329affe90c0f6fca108619e44afb4cdf30ec0fb24fcec6: Status 404 returned error can't find the container with id cba2b2889e05e1dcb9329affe90c0f6fca108619e44afb4cdf30ec0fb24fcec6 Sep 08 08:10:53.323854 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:53.323829 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/69650628-eaa4-400d-9c77-eacece44749a-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-5ncz8\" (UID: \"69650628-eaa4-400d-9c77-eacece44749a\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-5ncz8" Sep 08 08:10:53.323950 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:53.323866 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djvz2\" (UniqueName: \"kubernetes.io/projected/69650628-eaa4-400d-9c77-eacece44749a-kube-api-access-djvz2\") pod \"monitoring-plugin-85bbbbbf4-5ncz8\" (UID: \"69650628-eaa4-400d-9c77-eacece44749a\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-5ncz8" Sep 08 08:10:53.326304 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:53.326283 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/69650628-eaa4-400d-9c77-eacece44749a-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-5ncz8\" (UID: \"69650628-eaa4-400d-9c77-eacece44749a\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-5ncz8" Sep 08 08:10:53.332920 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:53.332901 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djvz2\" (UniqueName: \"kubernetes.io/projected/69650628-eaa4-400d-9c77-eacece44749a-kube-api-access-djvz2\") pod \"monitoring-plugin-85bbbbbf4-5ncz8\" (UID: \"69650628-eaa4-400d-9c77-eacece44749a\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-5ncz8" Sep 08 08:10:53.361356 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:53.361327 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mczjr"] Sep 08 08:10:53.366418 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:10:53.366365 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddac5eaa3_6c21_4b05_ace5_a13c8344ab6e.slice/crio-ea898e06b0cabb6328f94d4217910770085b730027844cc22ca3b83b0a9eb6d3 WatchSource:0}: Error finding container ea898e06b0cabb6328f94d4217910770085b730027844cc22ca3b83b0a9eb6d3: Status 404 returned error can't find the container with id ea898e06b0cabb6328f94d4217910770085b730027844cc22ca3b83b0a9eb6d3 Sep 08 08:10:53.388312 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:53.388288 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-k45rp"] Sep 08 08:10:53.391804 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:10:53.391775 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7301238b_b7d9_4101_8f7a_5e442499ca05.slice/crio-d6944fe43aa2564c484cedb7f9a4ec2d15a316e649616f77c2c50b7b230f579a WatchSource:0}: Error finding container d6944fe43aa2564c484cedb7f9a4ec2d15a316e649616f77c2c50b7b230f579a: Status 404 returned error can't find the container with id d6944fe43aa2564c484cedb7f9a4ec2d15a316e649616f77c2c50b7b230f579a Sep 08 08:10:53.499308 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:53.499283 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-5ncz8" Sep 08 08:10:53.506031 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:53.506009 2585 generic.go:356] "Generic (PLEG): container finished" podID="7932f481-a24e-4554-9401-b69cf440085e" containerID="fc7ab48b575f1b2d46043fd3b998fc5d23da4b57e587fb03639d3515a5957712" exitCode=0 Sep 08 08:10:53.506115 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:53.506094 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7932f481-a24e-4554-9401-b69cf440085e","Type":"ContainerDied","Data":"fc7ab48b575f1b2d46043fd3b998fc5d23da4b57e587fb03639d3515a5957712"} Sep 08 08:10:53.507420 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:53.507401 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-k45rp" event={"ID":"7301238b-b7d9-4101-8f7a-5e442499ca05","Type":"ContainerStarted","Data":"d6944fe43aa2564c484cedb7f9a4ec2d15a316e649616f77c2c50b7b230f579a"} Sep 08 08:10:53.508586 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:53.508564 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mczjr" event={"ID":"dac5eaa3-6c21-4b05-ace5-a13c8344ab6e","Type":"ContainerStarted","Data":"ea898e06b0cabb6328f94d4217910770085b730027844cc22ca3b83b0a9eb6d3"} Sep 08 08:10:53.509811 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:53.509786 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ns5tq" event={"ID":"95940b17-787c-4892-96f0-83cf9501e4d5","Type":"ContainerStarted","Data":"cba2b2889e05e1dcb9329affe90c0f6fca108619e44afb4cdf30ec0fb24fcec6"} Sep 08 08:10:53.770621 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:53.770544 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-5ncz8"] Sep 08 08:10:54.344684 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:10:54.344651 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69650628_eaa4_400d_9c77_eacece44749a.slice/crio-d854f2651f28e10cfcfbc6ad414b4b33430ad74ac97fd8439d9d10d2d00b2336 WatchSource:0}: Error finding container d854f2651f28e10cfcfbc6ad414b4b33430ad74ac97fd8439d9d10d2d00b2336: Status 404 returned error can't find the container with id d854f2651f28e10cfcfbc6ad414b4b33430ad74ac97fd8439d9d10d2d00b2336 Sep 08 08:10:54.518639 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.518550 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-5ncz8" event={"ID":"69650628-eaa4-400d-9c77-eacece44749a","Type":"ContainerStarted","Data":"d854f2651f28e10cfcfbc6ad414b4b33430ad74ac97fd8439d9d10d2d00b2336"} Sep 08 08:10:54.522727 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.522518 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7699cb4d56-j2wp8" event={"ID":"edebb973-5c2f-4951-971b-eb30724528c9","Type":"ContainerStarted","Data":"7671ce2b91cca7e53a0f3ab5b396f896761421ced6d665fa8bf23104b2509f4d"} Sep 08 08:10:54.765708 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.765143 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 08 08:10:54.771224 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.771162 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.775586 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.774561 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 08 08:10:54.775586 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.774758 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-4qxdn\"" Sep 08 08:10:54.775586 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.775435 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 08 08:10:54.778759 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.778726 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 08 08:10:54.779074 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.779054 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 08 08:10:54.779306 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.779286 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 08 08:10:54.779585 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.779570 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 08 08:10:54.780206 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.780189 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2bmqj4jvv5sss\"" Sep 08 08:10:54.780526 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.780479 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 08 08:10:54.781475 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.781457 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 08 08:10:54.781778 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.781736 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 08 08:10:54.781956 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.781930 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 08 08:10:54.783216 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.782860 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 08 08:10:54.783216 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.783092 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 08 08:10:54.783778 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.783638 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 08 08:10:54.788154 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.787957 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 08 08:10:54.789372 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.789353 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 08 08:10:54.790157 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.790133 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 08 08:10:54.838276 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.838156 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/581c35a0-bb28-4749-a432-8d4757b0abcf-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.838276 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.838204 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/581c35a0-bb28-4749-a432-8d4757b0abcf-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.838276 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.838232 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/581c35a0-bb28-4749-a432-8d4757b0abcf-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.838582 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.838290 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/581c35a0-bb28-4749-a432-8d4757b0abcf-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.838582 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.838315 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/581c35a0-bb28-4749-a432-8d4757b0abcf-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.838582 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.838349 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rbfn\" (UniqueName: \"kubernetes.io/projected/581c35a0-bb28-4749-a432-8d4757b0abcf-kube-api-access-7rbfn\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.838582 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.838394 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/581c35a0-bb28-4749-a432-8d4757b0abcf-config-out\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.838582 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.838418 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/581c35a0-bb28-4749-a432-8d4757b0abcf-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.838582 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.838441 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/581c35a0-bb28-4749-a432-8d4757b0abcf-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.838582 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.838461 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/581c35a0-bb28-4749-a432-8d4757b0abcf-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.838582 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.838502 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/581c35a0-bb28-4749-a432-8d4757b0abcf-web-config\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.838582 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.838529 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/581c35a0-bb28-4749-a432-8d4757b0abcf-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.838582 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.838558 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/581c35a0-bb28-4749-a432-8d4757b0abcf-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.839087 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.838585 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/581c35a0-bb28-4749-a432-8d4757b0abcf-config\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.839087 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.838609 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/581c35a0-bb28-4749-a432-8d4757b0abcf-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.839087 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.838640 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/581c35a0-bb28-4749-a432-8d4757b0abcf-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.839087 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.838664 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/581c35a0-bb28-4749-a432-8d4757b0abcf-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.839087 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.838699 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/581c35a0-bb28-4749-a432-8d4757b0abcf-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.839087 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.838732 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/581c35a0-bb28-4749-a432-8d4757b0abcf-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.839087 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.838762 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/581c35a0-bb28-4749-a432-8d4757b0abcf-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.940122 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.940080 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/581c35a0-bb28-4749-a432-8d4757b0abcf-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.940311 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.940130 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/581c35a0-bb28-4749-a432-8d4757b0abcf-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.940311 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.940159 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/581c35a0-bb28-4749-a432-8d4757b0abcf-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.940311 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.940195 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/581c35a0-bb28-4749-a432-8d4757b0abcf-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.940311 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.940218 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/581c35a0-bb28-4749-a432-8d4757b0abcf-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.940311 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.940243 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/581c35a0-bb28-4749-a432-8d4757b0abcf-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.940311 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.940267 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/581c35a0-bb28-4749-a432-8d4757b0abcf-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.940311 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.940287 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/581c35a0-bb28-4749-a432-8d4757b0abcf-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.940671 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.940319 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rbfn\" (UniqueName: \"kubernetes.io/projected/581c35a0-bb28-4749-a432-8d4757b0abcf-kube-api-access-7rbfn\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.940671 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.940351 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/581c35a0-bb28-4749-a432-8d4757b0abcf-config-out\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.940671 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.940374 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/581c35a0-bb28-4749-a432-8d4757b0abcf-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.940671 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.940623 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/581c35a0-bb28-4749-a432-8d4757b0abcf-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.940671 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.940650 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/581c35a0-bb28-4749-a432-8d4757b0abcf-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.940928 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.940694 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/581c35a0-bb28-4749-a432-8d4757b0abcf-web-config\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.940928 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.940722 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/581c35a0-bb28-4749-a432-8d4757b0abcf-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.940928 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.940749 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/581c35a0-bb28-4749-a432-8d4757b0abcf-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.940928 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.940777 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/581c35a0-bb28-4749-a432-8d4757b0abcf-config\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.940928 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.940800 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/581c35a0-bb28-4749-a432-8d4757b0abcf-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.940928 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.940808 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/581c35a0-bb28-4749-a432-8d4757b0abcf-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.940928 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.940841 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/581c35a0-bb28-4749-a432-8d4757b0abcf-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.940928 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.940867 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/581c35a0-bb28-4749-a432-8d4757b0abcf-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.941315 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.941207 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/581c35a0-bb28-4749-a432-8d4757b0abcf-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.941910 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.941884 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/581c35a0-bb28-4749-a432-8d4757b0abcf-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.943155 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.942814 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/581c35a0-bb28-4749-a432-8d4757b0abcf-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.945534 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.945508 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/581c35a0-bb28-4749-a432-8d4757b0abcf-config-out\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.945950 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.945836 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/581c35a0-bb28-4749-a432-8d4757b0abcf-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.946512 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.946487 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/581c35a0-bb28-4749-a432-8d4757b0abcf-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.947570 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.946702 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/581c35a0-bb28-4749-a432-8d4757b0abcf-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.947570 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.947172 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/581c35a0-bb28-4749-a432-8d4757b0abcf-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.949955 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.949636 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/581c35a0-bb28-4749-a432-8d4757b0abcf-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.951682 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.950570 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/581c35a0-bb28-4749-a432-8d4757b0abcf-web-config\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.951682 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.950793 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/581c35a0-bb28-4749-a432-8d4757b0abcf-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.951682 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.951170 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/581c35a0-bb28-4749-a432-8d4757b0abcf-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.951682 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.951643 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/581c35a0-bb28-4749-a432-8d4757b0abcf-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.952969 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.952709 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/581c35a0-bb28-4749-a432-8d4757b0abcf-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.952969 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.952929 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/581c35a0-bb28-4749-a432-8d4757b0abcf-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.953110 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.952975 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/581c35a0-bb28-4749-a432-8d4757b0abcf-config\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.954432 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.954070 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/581c35a0-bb28-4749-a432-8d4757b0abcf-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.954432 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.954366 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/581c35a0-bb28-4749-a432-8d4757b0abcf-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:54.955151 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:54.955104 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rbfn\" (UniqueName: \"kubernetes.io/projected/581c35a0-bb28-4749-a432-8d4757b0abcf-kube-api-access-7rbfn\") pod \"prometheus-k8s-0\" (UID: \"581c35a0-bb28-4749-a432-8d4757b0abcf\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:55.088073 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:55.087781 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:10:55.551394 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:55.551240 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7699cb4d56-j2wp8" event={"ID":"edebb973-5c2f-4951-971b-eb30724528c9","Type":"ContainerStarted","Data":"097400fef1e0b2578d7b02393e910273d1d588beb391bc5d17e9df65ef41d2d9"} Sep 08 08:10:55.551394 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:55.551325 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7699cb4d56-j2wp8" event={"ID":"edebb973-5c2f-4951-971b-eb30724528c9","Type":"ContainerStarted","Data":"80df6f929a3145b6d0ed40829459157aa5ca57f031ef91e1619328e58f32fa61"} Sep 08 08:10:55.779485 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:55.778740 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 08 08:10:56.125217 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:10:56.125184 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod581c35a0_bb28_4749_a432_8d4757b0abcf.slice/crio-526cd5e2605ae1c4eed8400e38bc50a6b0d5fef1e605610618e89aa459e56eb9 WatchSource:0}: Error finding container 526cd5e2605ae1c4eed8400e38bc50a6b0d5fef1e605610618e89aa459e56eb9: Status 404 returned error can't find the container with id 526cd5e2605ae1c4eed8400e38bc50a6b0d5fef1e605610618e89aa459e56eb9 Sep 08 08:10:56.559522 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:56.559428 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ns5tq" event={"ID":"95940b17-787c-4892-96f0-83cf9501e4d5","Type":"ContainerStarted","Data":"13d98b1814ec11bf7efb7d48cc08b876bbde196a509535262bfa4793ae6c2091"} Sep 08 08:10:56.562160 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:56.562130 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"581c35a0-bb28-4749-a432-8d4757b0abcf","Type":"ContainerStarted","Data":"526cd5e2605ae1c4eed8400e38bc50a6b0d5fef1e605610618e89aa459e56eb9"} Sep 08 08:10:57.568389 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:57.568354 2585 generic.go:356] "Generic (PLEG): container finished" podID="581c35a0-bb28-4749-a432-8d4757b0abcf" containerID="579ca0e38cd0dad0ecc30aaf09aeb9f02cb40a9ba88d312548be281e76171153" exitCode=0 Sep 08 08:10:57.568852 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:57.568504 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"581c35a0-bb28-4749-a432-8d4757b0abcf","Type":"ContainerDied","Data":"579ca0e38cd0dad0ecc30aaf09aeb9f02cb40a9ba88d312548be281e76171153"} Sep 08 08:10:57.571454 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:57.571422 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7932f481-a24e-4554-9401-b69cf440085e","Type":"ContainerStarted","Data":"28618e0964fcc530b4021e5bf48d3583dede69cdbcf2082bafef2fc88c58e2c4"} Sep 08 08:10:57.573778 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:57.573746 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ns5tq" event={"ID":"95940b17-787c-4892-96f0-83cf9501e4d5","Type":"ContainerStarted","Data":"e17869a80055175cd26491741bf496baa0e1030f37406e428bc21025e9828e4c"} Sep 08 08:10:57.576181 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:57.576159 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-5ncz8" event={"ID":"69650628-eaa4-400d-9c77-eacece44749a","Type":"ContainerStarted","Data":"1d6aa0fee7491af93edb05955166bf230e2ec949fcdf1707a3b93550b7071220"} Sep 08 08:10:57.576876 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:57.576852 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-5ncz8" Sep 08 08:10:57.583008 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:57.582990 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-5ncz8" Sep 08 08:10:57.618083 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:57.618034 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ns5tq" podStartSLOduration=68.398357021 podStartE2EDuration="1m10.618021448s" totalImagesPullingTime="2.219664427s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 08:09:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:10:57.617203772 +0000 UTC m=+70.954843415" watchObservedRunningTime="2026-09-08 08:10:57.618021448 +0000 UTC m=+70.955661089" Sep 08 08:10:58.583993 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:58.583910 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7932f481-a24e-4554-9401-b69cf440085e","Type":"ContainerStarted","Data":"8073cfd1c27afe84735ac7f40337be00378147f10d6a1cbfbf52cad4a87d71d2"} Sep 08 08:10:59.594533 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:59.594359 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7932f481-a24e-4554-9401-b69cf440085e","Type":"ContainerStarted","Data":"d5901c284ae60c694e0b604759d0913b0fa12c6e2413bc6299051e48cfdea4a3"} Sep 08 08:10:59.596703 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:59.596656 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-k45rp" event={"ID":"7301238b-b7d9-4101-8f7a-5e442499ca05","Type":"ContainerStarted","Data":"2c63655db40076003a943450893a5408fef38e989705afd774de99bd32d0cbfe"} Sep 08 08:10:59.597362 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:59.596939 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:10:59.602391 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:59.602329 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7699cb4d56-j2wp8" event={"ID":"edebb973-5c2f-4951-971b-eb30724528c9","Type":"ContainerStarted","Data":"42507fe89db3486a2a68a8b2644d1fda30442de7bb56e19f87dca34c7c0598f8"} Sep 08 08:10:59.614341 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:59.613612 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-k45rp" podStartSLOduration=65.535074088 podStartE2EDuration="1m11.613595189s" totalImagesPullingTime="6.078521101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 08:09:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:10:59.612455007 +0000 UTC m=+72.950094650" watchObservedRunningTime="2026-09-08 08:10:59.613595189 +0000 UTC m=+72.951234832" Sep 08 08:10:59.614341 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:10:59.614004 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-5ncz8" podStartSLOduration=4.35328884 podStartE2EDuration="6.613995363s" totalImagesPullingTime="2.260706523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 08:10:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:10:57.635436788 +0000 UTC m=+70.973076431" watchObservedRunningTime="2026-09-08 08:10:59.613995363 +0000 UTC m=+72.951635004" Sep 08 08:11:00.614322 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:11:00.614285 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7932f481-a24e-4554-9401-b69cf440085e","Type":"ContainerStarted","Data":"48253f754a3e6eecd20f2dafe84c10ed1997e478722b94621357a19cb698e9d4"} Sep 08 08:11:00.614779 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:11:00.614391 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7932f481-a24e-4554-9401-b69cf440085e","Type":"ContainerStarted","Data":"ff1848cbf34b5c5ea9da02d786b511afb64bd40b36aec9d492ac1c05d96b198c"} Sep 08 08:11:00.614779 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:11:00.614433 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7932f481-a24e-4554-9401-b69cf440085e","Type":"ContainerStarted","Data":"df07a87c00ea6fe3835bb558a8b53cbc9ac98a754875849e516ee7638b502657"} Sep 08 08:11:00.616996 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:11:00.616963 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mczjr" event={"ID":"dac5eaa3-6c21-4b05-ace5-a13c8344ab6e","Type":"ContainerStarted","Data":"f30e0af7c4739ab82fe66dda91aa99375d217a02e587365e03680f6a6932e57c"} Sep 08 08:11:00.621162 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:11:00.621133 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7699cb4d56-j2wp8" event={"ID":"edebb973-5c2f-4951-971b-eb30724528c9","Type":"ContainerStarted","Data":"9004ea2188a66dce4e4310336e9e6aec54dc24bd40a00ff67f61b2f12ca5386c"} Sep 08 08:11:00.621272 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:11:00.621200 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7699cb4d56-j2wp8" event={"ID":"edebb973-5c2f-4951-971b-eb30724528c9","Type":"ContainerStarted","Data":"aeabeb4d6650d4ee8b96f1d34993fb1cc03be6c669e6534a546610f0f9293693"} Sep 08 08:11:00.621691 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:11:00.621674 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7699cb4d56-j2wp8" Sep 08 08:11:00.628607 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:11:00.628588 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7699cb4d56-j2wp8" Sep 08 08:11:00.641169 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:11:00.641131 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=6.472378126 podStartE2EDuration="11.641116383s" totalImagesPullingTime="5.145370028s" totalInitContainerRuntime="23.368229ms" isStatefulPod=false podCreationTimestamp="2026-09-08 08:10:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:11:00.640485627 +0000 UTC m=+73.978125271" watchObservedRunningTime="2026-09-08 08:11:00.641116383 +0000 UTC m=+73.978756026" Sep 08 08:11:00.664103 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:11:00.664049 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-mczjr" podStartSLOduration=66.574323072 podStartE2EDuration="1m12.664027722s" totalImagesPullingTime="6.08970465s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 08:09:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:11:00.663021519 +0000 UTC m=+74.000661162" watchObservedRunningTime="2026-09-08 08:11:00.664027722 +0000 UTC m=+74.001667368" Sep 08 08:11:00.687490 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:11:00.686835 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7699cb4d56-j2wp8" podStartSLOduration=2.586298707 podStartE2EDuration="9.686815198s" totalImagesPullingTime="7.100516491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 08:10:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:11:00.685114126 +0000 UTC m=+74.022753768" watchObservedRunningTime="2026-09-08 08:11:00.686815198 +0000 UTC m=+74.024454840" Sep 08 08:11:01.651926 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:11:01.651658 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"581c35a0-bb28-4749-a432-8d4757b0abcf","Type":"ContainerStarted","Data":"03dfc1ab65d8e4e81f268f1b265fe987660ccb72aa05fbed98aa91112850f366"} Sep 08 08:11:01.651926 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:11:01.651777 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"581c35a0-bb28-4749-a432-8d4757b0abcf","Type":"ContainerStarted","Data":"0b0b6b13aadb7af6097a994c7a1eb382ec15c4d211fdae3fbaa1e523bd736d57"} Sep 08 08:11:01.651926 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:11:01.651816 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"581c35a0-bb28-4749-a432-8d4757b0abcf","Type":"ContainerStarted","Data":"c68229719c24c914907732fcaaddda8b770a76266b67cc18c75de2bc5f17c397"} Sep 08 08:11:02.659151 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:11:02.659120 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"581c35a0-bb28-4749-a432-8d4757b0abcf","Type":"ContainerStarted","Data":"27de7b485ff9b8c36d2a5a7c9834b559fb22bd18557080f79259e49a0370fd1e"} Sep 08 08:11:02.659513 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:11:02.659180 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"581c35a0-bb28-4749-a432-8d4757b0abcf","Type":"ContainerStarted","Data":"9b590956508b3ccbedcc9c451e4b095ee245fe0552e0cf020a55ff4b63b63239"} Sep 08 08:11:02.659513 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:11:02.659209 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"581c35a0-bb28-4749-a432-8d4757b0abcf","Type":"ContainerStarted","Data":"ffe3afe651124da3f4cd038a9454738abcedf5e59047cb3e1837e225bfda1834"} Sep 08 08:11:02.692320 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:11:02.692278 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.848776026 podStartE2EDuration="8.692255605s" totalImagesPullingTime="3.809324196s" totalInitContainerRuntime="34.155383ms" isStatefulPod=false podCreationTimestamp="2026-09-08 08:10:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:11:02.690266549 +0000 UTC m=+76.027906191" watchObservedRunningTime="2026-09-08 08:11:02.692255605 +0000 UTC m=+76.029895271" Sep 08 08:11:05.088412 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:11:05.088367 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:11:30.624099 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:11:30.623979 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-k45rp" Sep 08 08:11:55.088698 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:11:55.088662 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:11:55.107779 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:11:55.107756 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:11:55.879496 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:11:55.879475 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:14:47.034627 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:14:47.034591 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2xlvk_e00cae51-f771-4039-81ec-54fa2a48e98f/ovn-acl-logging/0.log" Sep 08 08:14:47.036092 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:14:47.036071 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2xlvk_e00cae51-f771-4039-81ec-54fa2a48e98f/ovn-acl-logging/0.log" Sep 08 08:14:47.042119 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:14:47.042102 2585 kubelet.go:1648] "Image garbage collection succeeded" Sep 08 08:19:47.062465 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:19:47.062434 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2xlvk_e00cae51-f771-4039-81ec-54fa2a48e98f/ovn-acl-logging/0.log" Sep 08 08:19:47.063331 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:19:47.063315 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2xlvk_e00cae51-f771-4039-81ec-54fa2a48e98f/ovn-acl-logging/0.log" Sep 08 08:21:14.049111 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:21:14.049068 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-test-apply-drhvq"] Sep 08 08:21:14.051130 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:21:14.051115 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-drhvq" Sep 08 08:21:14.053581 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:21:14.053553 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 08 08:21:14.053719 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:21:14.053553 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 08 08:21:14.053719 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:21:14.053587 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-cronjob-dockercfg-qz82d\"" Sep 08 08:21:14.054357 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:21:14.054334 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Sep 08 08:21:14.059916 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:21:14.059894 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-test-apply-drhvq"] Sep 08 08:21:14.172175 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:21:14.172145 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgx2r\" (UniqueName: \"kubernetes.io/projected/624b15f8-ceea-44f7-a359-1a27e1f5c6f8-kube-api-access-mgx2r\") pod \"feast-test-apply-drhvq\" (UID: \"624b15f8-ceea-44f7-a359-1a27e1f5c6f8\") " pod="test-ns-feast/feast-test-apply-drhvq" Sep 08 08:21:14.273484 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:21:14.273444 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgx2r\" (UniqueName: \"kubernetes.io/projected/624b15f8-ceea-44f7-a359-1a27e1f5c6f8-kube-api-access-mgx2r\") pod \"feast-test-apply-drhvq\" (UID: \"624b15f8-ceea-44f7-a359-1a27e1f5c6f8\") " pod="test-ns-feast/feast-test-apply-drhvq" Sep 08 08:21:14.288911 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:21:14.288882 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgx2r\" (UniqueName: \"kubernetes.io/projected/624b15f8-ceea-44f7-a359-1a27e1f5c6f8-kube-api-access-mgx2r\") pod \"feast-test-apply-drhvq\" (UID: \"624b15f8-ceea-44f7-a359-1a27e1f5c6f8\") " pod="test-ns-feast/feast-test-apply-drhvq" Sep 08 08:21:14.372910 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:21:14.372825 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-drhvq" Sep 08 08:21:14.642983 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:21:14.642958 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-test-apply-drhvq"] Sep 08 08:21:14.645825 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:21:14.645796 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod624b15f8_ceea_44f7_a359_1a27e1f5c6f8.slice/crio-fef00ab18b726de54662a51197fe61d86e000ab16f9762057fb1e5e2d911420b WatchSource:0}: Error finding container fef00ab18b726de54662a51197fe61d86e000ab16f9762057fb1e5e2d911420b: Status 404 returned error can't find the container with id fef00ab18b726de54662a51197fe61d86e000ab16f9762057fb1e5e2d911420b Sep 08 08:21:14.648112 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:21:14.648094 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 08 08:21:14.877516 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:21:14.877484 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-drhvq" event={"ID":"624b15f8-ceea-44f7-a359-1a27e1f5c6f8","Type":"ContainerStarted","Data":"fef00ab18b726de54662a51197fe61d86e000ab16f9762057fb1e5e2d911420b"} Sep 08 08:21:19.905738 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:21:19.905703 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-drhvq" event={"ID":"624b15f8-ceea-44f7-a359-1a27e1f5c6f8","Type":"ContainerStarted","Data":"1b6bfad64f6915df40a3d2d5577bc93eb08ea141870a58a5741d9a343b657f9b"} Sep 08 08:21:26.934425 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:21:26.934324 2585 generic.go:356] "Generic (PLEG): container finished" podID="624b15f8-ceea-44f7-a359-1a27e1f5c6f8" containerID="1b6bfad64f6915df40a3d2d5577bc93eb08ea141870a58a5741d9a343b657f9b" exitCode=0 Sep 08 08:21:26.934425 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:21:26.934407 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-drhvq" event={"ID":"624b15f8-ceea-44f7-a359-1a27e1f5c6f8","Type":"ContainerDied","Data":"1b6bfad64f6915df40a3d2d5577bc93eb08ea141870a58a5741d9a343b657f9b"} Sep 08 08:21:27.940506 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:21:27.940468 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-drhvq" event={"ID":"624b15f8-ceea-44f7-a359-1a27e1f5c6f8","Type":"ContainerStarted","Data":"4f31e817839d2807f0c6c663a23f61e6bd68cfe1fe1e381577d6538e0c19ef48"} Sep 08 08:21:27.957571 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:21:27.957430 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-test-apply-drhvq" podStartSLOduration=1.999936478 podStartE2EDuration="13.957411548s" totalImagesPullingTime="4.759686439s" totalInitContainerRuntime="7.197788631s" isStatefulPod=false podCreationTimestamp="2026-09-08 08:21:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:21:27.956838851 +0000 UTC m=+701.294478494" watchObservedRunningTime="2026-09-08 08:21:27.957411548 +0000 UTC m=+701.295051191" Sep 08 08:21:37.980125 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:21:37.980087 2585 generic.go:356] "Generic (PLEG): container finished" podID="624b15f8-ceea-44f7-a359-1a27e1f5c6f8" containerID="4f31e817839d2807f0c6c663a23f61e6bd68cfe1fe1e381577d6538e0c19ef48" exitCode=0 Sep 08 08:21:37.980517 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:21:37.980166 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-drhvq" event={"ID":"624b15f8-ceea-44f7-a359-1a27e1f5c6f8","Type":"ContainerDied","Data":"4f31e817839d2807f0c6c663a23f61e6bd68cfe1fe1e381577d6538e0c19ef48"} Sep 08 08:21:39.111844 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:21:39.111823 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-drhvq" Sep 08 08:21:39.178513 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:21:39.178483 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mgx2r\" (UniqueName: \"kubernetes.io/projected/624b15f8-ceea-44f7-a359-1a27e1f5c6f8-kube-api-access-mgx2r\") pod \"624b15f8-ceea-44f7-a359-1a27e1f5c6f8\" (UID: \"624b15f8-ceea-44f7-a359-1a27e1f5c6f8\") " Sep 08 08:21:39.180734 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:21:39.180706 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/624b15f8-ceea-44f7-a359-1a27e1f5c6f8-kube-api-access-mgx2r" (OuterVolumeSpecName: "kube-api-access-mgx2r") pod "624b15f8-ceea-44f7-a359-1a27e1f5c6f8" (UID: "624b15f8-ceea-44f7-a359-1a27e1f5c6f8"). InnerVolumeSpecName "kube-api-access-mgx2r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 08:21:39.279025 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:21:39.278938 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mgx2r\" (UniqueName: \"kubernetes.io/projected/624b15f8-ceea-44f7-a359-1a27e1f5c6f8-kube-api-access-mgx2r\") on node \"ip-10-0-143-1.ec2.internal\" DevicePath \"\"" Sep 08 08:21:39.989569 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:21:39.989542 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-drhvq" Sep 08 08:21:39.989728 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:21:39.989541 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-drhvq" event={"ID":"624b15f8-ceea-44f7-a359-1a27e1f5c6f8","Type":"ContainerDied","Data":"fef00ab18b726de54662a51197fe61d86e000ab16f9762057fb1e5e2d911420b"} Sep 08 08:21:39.989728 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:21:39.989655 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fef00ab18b726de54662a51197fe61d86e000ab16f9762057fb1e5e2d911420b" Sep 08 08:21:40.163185 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:21:40.163144 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-feast_feast-test-apply-drhvq_624b15f8-ceea-44f7-a359-1a27e1f5c6f8/feast-0/0.log" Sep 08 08:21:40.169171 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:21:40.169148 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-feast_feast-test-apply-drhvq_624b15f8-ceea-44f7-a359-1a27e1f5c6f8/feast-1/0.log" Sep 08 08:22:20.916043 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:22:20.916011 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-test-apply-drhvq"] Sep 08 08:22:20.922218 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:22:20.922194 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-test-apply-drhvq"] Sep 08 08:22:21.173729 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:22:21.173663 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="624b15f8-ceea-44f7-a359-1a27e1f5c6f8" path="/var/lib/kubelet/pods/624b15f8-ceea-44f7-a359-1a27e1f5c6f8/volumes" Sep 08 08:24:47.088518 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:24:47.088446 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2xlvk_e00cae51-f771-4039-81ec-54fa2a48e98f/ovn-acl-logging/0.log" Sep 08 08:24:47.095026 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:24:47.095004 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2xlvk_e00cae51-f771-4039-81ec-54fa2a48e98f/ovn-acl-logging/0.log" Sep 08 08:27:31.201814 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:31.201783 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-fm56j"] Sep 08 08:27:31.202223 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:31.202097 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="624b15f8-ceea-44f7-a359-1a27e1f5c6f8" containerName="feast-1" Sep 08 08:27:31.202223 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:31.202110 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="624b15f8-ceea-44f7-a359-1a27e1f5c6f8" containerName="feast-1" Sep 08 08:27:31.202223 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:31.202124 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="624b15f8-ceea-44f7-a359-1a27e1f5c6f8" containerName="feast-0" Sep 08 08:27:31.202223 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:31.202134 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="624b15f8-ceea-44f7-a359-1a27e1f5c6f8" containerName="feast-0" Sep 08 08:27:31.202223 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:31.202201 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="624b15f8-ceea-44f7-a359-1a27e1f5c6f8" containerName="feast-1" Sep 08 08:27:31.205152 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:31.205136 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-fm56j" Sep 08 08:27:31.207681 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:31.207652 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Sep 08 08:27:31.207826 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:31.207654 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-dgk5s\"" Sep 08 08:27:31.208501 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:31.208485 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Sep 08 08:27:31.220234 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:31.220208 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-fm56j"] Sep 08 08:27:31.356306 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:31.356265 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qp4zg\" (UniqueName: \"kubernetes.io/projected/1fce6ff0-388f-4c13-9a83-9c7ece9b557b-kube-api-access-qp4zg\") pod \"feast-operator-controller-manager-689b877b4b-fm56j\" (UID: \"1fce6ff0-388f-4c13-9a83-9c7ece9b557b\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-fm56j" Sep 08 08:27:31.457299 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:31.457221 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qp4zg\" (UniqueName: \"kubernetes.io/projected/1fce6ff0-388f-4c13-9a83-9c7ece9b557b-kube-api-access-qp4zg\") pod \"feast-operator-controller-manager-689b877b4b-fm56j\" (UID: \"1fce6ff0-388f-4c13-9a83-9c7ece9b557b\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-fm56j" Sep 08 08:27:31.467803 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:31.467768 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qp4zg\" (UniqueName: \"kubernetes.io/projected/1fce6ff0-388f-4c13-9a83-9c7ece9b557b-kube-api-access-qp4zg\") pod \"feast-operator-controller-manager-689b877b4b-fm56j\" (UID: \"1fce6ff0-388f-4c13-9a83-9c7ece9b557b\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-fm56j" Sep 08 08:27:31.515472 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:31.515430 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-fm56j" Sep 08 08:27:31.785974 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:31.785904 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-fm56j"] Sep 08 08:27:31.788632 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:27:31.788592 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fce6ff0_388f_4c13_9a83_9c7ece9b557b.slice/crio-91741f61aa09d904d2120f56b760a43a3e63bc86967f95e2c2801c4021da5f05 WatchSource:0}: Error finding container 91741f61aa09d904d2120f56b760a43a3e63bc86967f95e2c2801c4021da5f05: Status 404 returned error can't find the container with id 91741f61aa09d904d2120f56b760a43a3e63bc86967f95e2c2801c4021da5f05 Sep 08 08:27:31.790475 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:31.790455 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 08 08:27:32.310322 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:32.310288 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-fm56j" event={"ID":"1fce6ff0-388f-4c13-9a83-9c7ece9b557b","Type":"ContainerStarted","Data":"91741f61aa09d904d2120f56b760a43a3e63bc86967f95e2c2801c4021da5f05"} Sep 08 08:27:35.326248 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:35.326212 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-fm56j" event={"ID":"1fce6ff0-388f-4c13-9a83-9c7ece9b557b","Type":"ContainerStarted","Data":"1a7fb0dd17a08da1a47c65aa07f1a23697509c7a940e62710a3125399a781375"} Sep 08 08:27:35.326638 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:35.326329 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-fm56j" Sep 08 08:27:35.345057 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:35.345006 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-fm56j" podStartSLOduration=1.835361517 podStartE2EDuration="4.344988929s" totalImagesPullingTime="2.509627412s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 08:27:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:27:35.343683852 +0000 UTC m=+1068.681323495" watchObservedRunningTime="2026-09-08 08:27:35.344988929 +0000 UTC m=+1068.682628570" Sep 08 08:27:46.332611 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:46.332531 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-fm56j" Sep 08 08:27:47.114104 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:47.114070 2585 scope.go:117] "RemoveContainer" containerID="4f31e817839d2807f0c6c663a23f61e6bd68cfe1fe1e381577d6538e0c19ef48" Sep 08 08:27:47.122611 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:47.122593 2585 scope.go:117] "RemoveContainer" containerID="1b6bfad64f6915df40a3d2d5577bc93eb08ea141870a58a5741d9a343b657f9b" Sep 08 08:27:48.167279 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:48.167238 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh"] Sep 08 08:27:48.174358 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:48.174332 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:27:48.176805 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:48.176779 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 08 08:27:48.176953 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:48.176805 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 08 08:27:48.177237 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:48.177221 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 08 08:27:48.177309 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:48.177271 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-r6zwb\"" Sep 08 08:27:48.177309 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:48.177280 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 08 08:27:48.177309 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:48.177291 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 08 08:27:48.177642 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:48.177629 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 08 08:27:48.183180 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:48.183159 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh"] Sep 08 08:27:48.191366 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:48.191328 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-npczh\" (UID: \"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:27:48.191535 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:48.191414 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhlts\" (UniqueName: \"kubernetes.io/projected/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-kube-api-access-lhlts\") pod \"feast-simple-feast-setup-57576b8fcf-npczh\" (UID: \"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:27:48.191535 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:48.191457 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-npczh\" (UID: \"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:27:48.191535 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:48.191489 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-npczh\" (UID: \"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:27:48.191535 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:48.191518 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-npczh\" (UID: \"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:27:48.191766 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:48.191543 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-npczh\" (UID: \"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:27:48.292591 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:48.292543 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-npczh\" (UID: \"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:27:48.292591 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:48.292593 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-npczh\" (UID: \"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:27:48.292837 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:48.292640 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhlts\" (UniqueName: \"kubernetes.io/projected/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-kube-api-access-lhlts\") pod \"feast-simple-feast-setup-57576b8fcf-npczh\" (UID: \"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:27:48.292837 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:48.292673 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-npczh\" (UID: \"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:27:48.292837 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:27:48.292677 2585 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Sep 08 08:27:48.292837 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:48.292713 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-npczh\" (UID: \"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:27:48.292837 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:48.292748 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-npczh\" (UID: \"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:27:48.292837 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:27:48.292774 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-offline-tls podName:bd3a0e5c-9c97-4ff1-9c2f-3807405c608d nodeName:}" failed. No retries permitted until 2026-09-08 08:27:48.792747829 +0000 UTC m=+1082.130387449 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-offline-tls") pod "feast-simple-feast-setup-57576b8fcf-npczh" (UID: "bd3a0e5c-9c97-4ff1-9c2f-3807405c608d") : secret "feast-simple-feast-setup-offline-tls" not found Sep 08 08:27:48.293130 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:27:48.292859 2585 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Sep 08 08:27:48.293130 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:27:48.292917 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-ui-tls podName:bd3a0e5c-9c97-4ff1-9c2f-3807405c608d nodeName:}" failed. No retries permitted until 2026-09-08 08:27:48.792899345 +0000 UTC m=+1082.130538983 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-ui-tls") pod "feast-simple-feast-setup-57576b8fcf-npczh" (UID: "bd3a0e5c-9c97-4ff1-9c2f-3807405c608d") : secret "feast-simple-feast-setup-ui-tls" not found Sep 08 08:27:48.293213 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:48.293134 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-npczh\" (UID: \"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:27:48.295345 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:48.295311 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-npczh\" (UID: \"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:27:48.295509 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:48.295357 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-npczh\" (UID: \"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:27:48.300699 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:48.300675 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhlts\" (UniqueName: \"kubernetes.io/projected/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-kube-api-access-lhlts\") pod \"feast-simple-feast-setup-57576b8fcf-npczh\" (UID: \"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:27:48.796609 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:48.796575 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-npczh\" (UID: \"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:27:48.796609 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:48.796621 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-npczh\" (UID: \"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:27:48.796839 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:27:48.796724 2585 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Sep 08 08:27:48.796839 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:27:48.796783 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-offline-tls podName:bd3a0e5c-9c97-4ff1-9c2f-3807405c608d nodeName:}" failed. No retries permitted until 2026-09-08 08:27:49.79676943 +0000 UTC m=+1083.134409048 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-offline-tls") pod "feast-simple-feast-setup-57576b8fcf-npczh" (UID: "bd3a0e5c-9c97-4ff1-9c2f-3807405c608d") : secret "feast-simple-feast-setup-offline-tls" not found Sep 08 08:27:48.799212 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:48.799189 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-npczh\" (UID: \"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:27:49.805785 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:49.805754 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-npczh\" (UID: \"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:27:49.808543 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:49.808520 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-npczh\" (UID: \"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:27:49.999035 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:49.998975 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:27:50.285337 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:50.285307 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh"] Sep 08 08:27:50.289686 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:27:50.289653 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd3a0e5c_9c97_4ff1_9c2f_3807405c608d.slice/crio-0a560c3dccfae42ebd439f92f7b02b76fd2ce6b81a7d61849b98e56aee635571 WatchSource:0}: Error finding container 0a560c3dccfae42ebd439f92f7b02b76fd2ce6b81a7d61849b98e56aee635571: Status 404 returned error can't find the container with id 0a560c3dccfae42ebd439f92f7b02b76fd2ce6b81a7d61849b98e56aee635571 Sep 08 08:27:50.384572 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:27:50.384536 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" event={"ID":"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d","Type":"ContainerStarted","Data":"0a560c3dccfae42ebd439f92f7b02b76fd2ce6b81a7d61849b98e56aee635571"} Sep 08 08:28:04.455131 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:04.455090 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" event={"ID":"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d","Type":"ContainerStarted","Data":"8b71e7ada1fbcbc62044ffc1fab0777fa8ec3657b55ed440ee9298bb7c60b2ae"} Sep 08 08:28:09.477555 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:09.477521 2585 generic.go:356] "Generic (PLEG): container finished" podID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerID="8b71e7ada1fbcbc62044ffc1fab0777fa8ec3657b55ed440ee9298bb7c60b2ae" exitCode=0 Sep 08 08:28:09.477895 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:09.477598 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" event={"ID":"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d","Type":"ContainerDied","Data":"8b71e7ada1fbcbc62044ffc1fab0777fa8ec3657b55ed440ee9298bb7c60b2ae"} Sep 08 08:28:10.484004 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:10.483962 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" event={"ID":"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d","Type":"ContainerStarted","Data":"2bb1e7b07a09ed9ba824a296d45d177ee4cab512e2dfbd8854a0a769a2c5577e"} Sep 08 08:28:15.505652 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:15.505620 2585 generic.go:356] "Generic (PLEG): container finished" podID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerID="2bb1e7b07a09ed9ba824a296d45d177ee4cab512e2dfbd8854a0a769a2c5577e" exitCode=0 Sep 08 08:28:15.506098 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:15.505693 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" event={"ID":"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d","Type":"ContainerDied","Data":"2bb1e7b07a09ed9ba824a296d45d177ee4cab512e2dfbd8854a0a769a2c5577e"} Sep 08 08:28:16.516740 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:16.516693 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" event={"ID":"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d","Type":"ContainerStarted","Data":"67ef1817a42be560165ba5678402dcff5bbe0d0813646a4de9833c571cb32b0d"} Sep 08 08:28:16.517234 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:16.516767 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" event={"ID":"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d","Type":"ContainerStarted","Data":"5810be777a8a77d8dbbc968b1bbafc10ad054fc7af0a25fae6cd093522d578bb"} Sep 08 08:28:16.517234 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:16.516788 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" event={"ID":"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d","Type":"ContainerStarted","Data":"220aa93fadde761a75fbfce53ba4e645a1df39bf61c754b1ee95b0455c4764ee"} Sep 08 08:28:16.517234 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:16.516813 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" event={"ID":"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d","Type":"ContainerStarted","Data":"393c0681342cf481286d1ee544cd01114d47d103a99bfda8c049c7ffee98989a"} Sep 08 08:28:16.541350 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:16.541257 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" podStartSLOduration=14.825452603 podStartE2EDuration="28.541236331s" totalImagesPullingTime="13.715783728s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 08:27:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:28:16.54099906 +0000 UTC m=+1109.878638709" watchObservedRunningTime="2026-09-08 08:28:16.541236331 +0000 UTC m=+1109.878875974" Sep 08 08:28:16.999799 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:16.999750 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:28:17.000001 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:16.999821 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:28:17.000001 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:16.999842 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:28:17.000001 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:16.999861 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:28:17.002018 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:17.001949 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.17:6571: connect: connection refused" Sep 08 08:28:17.002165 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:17.002028 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.17:6567/health\": dial tcp 10.134.0.17:6567: connect: connection refused" Sep 08 08:28:17.002165 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:17.001973 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.17:8816: connect: connection refused" Sep 08 08:28:17.002165 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:17.002158 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.17:8443: connect: connection refused" Sep 08 08:28:19.999603 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:19.999562 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:28:20.000155 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:20.000117 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.17:6571: connect: connection refused" Sep 08 08:28:20.000415 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:20.000037 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.17:6567/health\": dial tcp 10.134.0.17:6567: connect: connection refused" Sep 08 08:28:20.000574 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:20.000041 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.17:8816: connect: connection refused" Sep 08 08:28:20.000700 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:20.000135 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:28:20.000769 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:20.000759 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:28:20.000821 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:20.000807 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:28:20.000919 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:19.999944 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.17:8443: connect: connection refused" Sep 08 08:28:23.000400 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:23.000322 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.17:8816: connect: connection refused" Sep 08 08:28:23.000876 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:23.000426 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.17:6571: connect: connection refused" Sep 08 08:28:23.000876 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:23.000484 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.17:8443: connect: connection refused" Sep 08 08:28:23.000876 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:23.000427 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.17:6567/health\": dial tcp 10.134.0.17:6567: connect: connection refused" Sep 08 08:28:26.001027 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:26.000995 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:28:26.001570 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:26.001547 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:28:26.001752 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:26.001734 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:28:26.001863 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:26.001801 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:28:26.006633 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:26.006615 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:28:26.569062 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:26.569022 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:28:26.569230 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:26.569205 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:28:26.571823 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:28:26.571807 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:29:28.583952 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:28.583869 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh"] Sep 08 08:29:28.584516 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:28.584100 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerName="registry" containerID="cri-o://393c0681342cf481286d1ee544cd01114d47d103a99bfda8c049c7ffee98989a" gracePeriod=30 Sep 08 08:29:28.584676 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:28.584645 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerName="offline" containerID="cri-o://5810be777a8a77d8dbbc968b1bbafc10ad054fc7af0a25fae6cd093522d578bb" gracePeriod=30 Sep 08 08:29:28.584821 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:28.584801 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerName="online" containerID="cri-o://220aa93fadde761a75fbfce53ba4e645a1df39bf61c754b1ee95b0455c4764ee" gracePeriod=30 Sep 08 08:29:28.584991 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:28.584150 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerName="ui" containerID="cri-o://67ef1817a42be560165ba5678402dcff5bbe0d0813646a4de9833c571cb32b0d" gracePeriod=30 Sep 08 08:29:29.037908 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:29.037862 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz"] Sep 08 08:29:29.040703 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:29.040680 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:29.042834 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:29.042809 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-wfc8k\"" Sep 08 08:29:29.058054 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:29.058031 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz"] Sep 08 08:29:29.113023 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:29.112974 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-fzfvz\" (UID: \"0e2d324f-2a5f-488a-9bc2-a0a56af7815b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:29.113023 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:29.113020 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rmnm\" (UniqueName: \"kubernetes.io/projected/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-kube-api-access-6rmnm\") pod \"feast-simple-feast-setup-57576b8fcf-fzfvz\" (UID: \"0e2d324f-2a5f-488a-9bc2-a0a56af7815b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:29.113259 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:29.113100 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fzfvz\" (UID: \"0e2d324f-2a5f-488a-9bc2-a0a56af7815b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:29.113259 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:29.113124 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fzfvz\" (UID: \"0e2d324f-2a5f-488a-9bc2-a0a56af7815b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:29.113259 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:29.113146 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fzfvz\" (UID: \"0e2d324f-2a5f-488a-9bc2-a0a56af7815b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:29.113259 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:29.113167 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fzfvz\" (UID: \"0e2d324f-2a5f-488a-9bc2-a0a56af7815b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:29.214478 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:29.214446 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fzfvz\" (UID: \"0e2d324f-2a5f-488a-9bc2-a0a56af7815b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:29.214689 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:29.214488 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fzfvz\" (UID: \"0e2d324f-2a5f-488a-9bc2-a0a56af7815b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:29.214689 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:29.214506 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fzfvz\" (UID: \"0e2d324f-2a5f-488a-9bc2-a0a56af7815b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:29.214689 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:29.214530 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fzfvz\" (UID: \"0e2d324f-2a5f-488a-9bc2-a0a56af7815b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:29.214689 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:29.214565 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-fzfvz\" (UID: \"0e2d324f-2a5f-488a-9bc2-a0a56af7815b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:29.214689 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:29.214585 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rmnm\" (UniqueName: \"kubernetes.io/projected/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-kube-api-access-6rmnm\") pod \"feast-simple-feast-setup-57576b8fcf-fzfvz\" (UID: \"0e2d324f-2a5f-488a-9bc2-a0a56af7815b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:29.214689 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:29:29.214659 2585 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found Sep 08 08:29:29.214959 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:29:29.214718 2585 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Sep 08 08:29:29.214959 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:29:29.214737 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-online-tls podName:0e2d324f-2a5f-488a-9bc2-a0a56af7815b nodeName:}" failed. No retries permitted until 2026-09-08 08:29:29.714715132 +0000 UTC m=+1183.052354751 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-online-tls") pod "feast-simple-feast-setup-57576b8fcf-fzfvz" (UID: "0e2d324f-2a5f-488a-9bc2-a0a56af7815b") : secret "feast-simple-feast-setup-online-tls" not found Sep 08 08:29:29.214959 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:29:29.214659 2585 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Sep 08 08:29:29.214959 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:29:29.214781 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-offline-tls podName:0e2d324f-2a5f-488a-9bc2-a0a56af7815b nodeName:}" failed. No retries permitted until 2026-09-08 08:29:29.714762735 +0000 UTC m=+1183.052402356 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-offline-tls") pod "feast-simple-feast-setup-57576b8fcf-fzfvz" (UID: "0e2d324f-2a5f-488a-9bc2-a0a56af7815b") : secret "feast-simple-feast-setup-offline-tls" not found Sep 08 08:29:29.214959 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:29:29.214799 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-registry-tls podName:0e2d324f-2a5f-488a-9bc2-a0a56af7815b nodeName:}" failed. No retries permitted until 2026-09-08 08:29:29.714790685 +0000 UTC m=+1183.052430303 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-registry-tls") pod "feast-simple-feast-setup-57576b8fcf-fzfvz" (UID: "0e2d324f-2a5f-488a-9bc2-a0a56af7815b") : secret "feast-simple-feast-setup-registry-tls" not found Sep 08 08:29:29.215124 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:29.214986 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-fzfvz\" (UID: \"0e2d324f-2a5f-488a-9bc2-a0a56af7815b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:29.217496 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:29.217461 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fzfvz\" (UID: \"0e2d324f-2a5f-488a-9bc2-a0a56af7815b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:29.225235 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:29.225208 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rmnm\" (UniqueName: \"kubernetes.io/projected/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-kube-api-access-6rmnm\") pod \"feast-simple-feast-setup-57576b8fcf-fzfvz\" (UID: \"0e2d324f-2a5f-488a-9bc2-a0a56af7815b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:29.718702 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:29.718673 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fzfvz\" (UID: \"0e2d324f-2a5f-488a-9bc2-a0a56af7815b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:29.718702 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:29.718711 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fzfvz\" (UID: \"0e2d324f-2a5f-488a-9bc2-a0a56af7815b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:29.719120 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:29.718726 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fzfvz\" (UID: \"0e2d324f-2a5f-488a-9bc2-a0a56af7815b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:29.721460 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:29.721437 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fzfvz\" (UID: \"0e2d324f-2a5f-488a-9bc2-a0a56af7815b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:29.721460 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:29.721452 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fzfvz\" (UID: \"0e2d324f-2a5f-488a-9bc2-a0a56af7815b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:29.721638 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:29.721606 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fzfvz\" (UID: \"0e2d324f-2a5f-488a-9bc2-a0a56af7815b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:29.815935 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:29.815900 2585 generic.go:356] "Generic (PLEG): container finished" podID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerID="67ef1817a42be560165ba5678402dcff5bbe0d0813646a4de9833c571cb32b0d" exitCode=0 Sep 08 08:29:29.815935 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:29.815930 2585 generic.go:356] "Generic (PLEG): container finished" podID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerID="220aa93fadde761a75fbfce53ba4e645a1df39bf61c754b1ee95b0455c4764ee" exitCode=0 Sep 08 08:29:29.816118 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:29.815967 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" event={"ID":"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d","Type":"ContainerDied","Data":"67ef1817a42be560165ba5678402dcff5bbe0d0813646a4de9833c571cb32b0d"} Sep 08 08:29:29.816118 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:29.816007 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" event={"ID":"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d","Type":"ContainerDied","Data":"220aa93fadde761a75fbfce53ba4e645a1df39bf61c754b1ee95b0455c4764ee"} Sep 08 08:29:29.949862 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:29.949831 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:30.227269 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:30.227244 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz"] Sep 08 08:29:30.228931 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:29:30.228909 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0e2d324f_2a5f_488a_9bc2_a0a56af7815b.slice/crio-4e4b3c6255edddf3cbfd439c1464201a42b16893397185aeee1a869cb5574cda WatchSource:0}: Error finding container 4e4b3c6255edddf3cbfd439c1464201a42b16893397185aeee1a869cb5574cda: Status 404 returned error can't find the container with id 4e4b3c6255edddf3cbfd439c1464201a42b16893397185aeee1a869cb5574cda Sep 08 08:29:30.821986 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:30.821950 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" event={"ID":"0e2d324f-2a5f-488a-9bc2-a0a56af7815b","Type":"ContainerStarted","Data":"fb83f7178ca832defa28c3cc24e52aa560e3782c48ab8976a92644d2fceb991d"} Sep 08 08:29:30.822363 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:30.821993 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" event={"ID":"0e2d324f-2a5f-488a-9bc2-a0a56af7815b","Type":"ContainerStarted","Data":"4e4b3c6255edddf3cbfd439c1464201a42b16893397185aeee1a869cb5574cda"} Sep 08 08:29:35.844455 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:35.844421 2585 generic.go:356] "Generic (PLEG): container finished" podID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerID="fb83f7178ca832defa28c3cc24e52aa560e3782c48ab8976a92644d2fceb991d" exitCode=0 Sep 08 08:29:35.844858 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:35.844490 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" event={"ID":"0e2d324f-2a5f-488a-9bc2-a0a56af7815b","Type":"ContainerDied","Data":"fb83f7178ca832defa28c3cc24e52aa560e3782c48ab8976a92644d2fceb991d"} Sep 08 08:29:36.568805 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:36.568764 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.17:6567/health\": dial tcp 10.134.0.17:6567: connect: connection refused" Sep 08 08:29:36.568989 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:36.568759 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.17:8443: connect: connection refused" Sep 08 08:29:36.850219 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:36.850139 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" event={"ID":"0e2d324f-2a5f-488a-9bc2-a0a56af7815b","Type":"ContainerStarted","Data":"7487bb1aa19fb506d8f6458101a1c000535df37cdb92969b4ec825411203c717"} Sep 08 08:29:41.872778 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:41.872745 2585 generic.go:356] "Generic (PLEG): container finished" podID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerID="7487bb1aa19fb506d8f6458101a1c000535df37cdb92969b4ec825411203c717" exitCode=0 Sep 08 08:29:41.873241 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:41.872824 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" event={"ID":"0e2d324f-2a5f-488a-9bc2-a0a56af7815b","Type":"ContainerDied","Data":"7487bb1aa19fb506d8f6458101a1c000535df37cdb92969b4ec825411203c717"} Sep 08 08:29:42.888501 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:42.888453 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" event={"ID":"0e2d324f-2a5f-488a-9bc2-a0a56af7815b","Type":"ContainerStarted","Data":"a9dd3978afa4a917c52353aea7e4b26c1de84ad375b96d288a0fdd2ae3d5f1a0"} Sep 08 08:29:42.888963 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:42.888515 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" event={"ID":"0e2d324f-2a5f-488a-9bc2-a0a56af7815b","Type":"ContainerStarted","Data":"3fd0179ef3ac841a69341c92512b880cd3dd350c6872a81d35af4db0daea4b73"} Sep 08 08:29:42.888963 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:42.888541 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" event={"ID":"0e2d324f-2a5f-488a-9bc2-a0a56af7815b","Type":"ContainerStarted","Data":"1b3287ee5d75dae31c1dd742e6bd412b4cf299d516bb7c8f77a4c34d2e3abfa2"} Sep 08 08:29:42.888963 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:42.888560 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" event={"ID":"0e2d324f-2a5f-488a-9bc2-a0a56af7815b","Type":"ContainerStarted","Data":"f4a1e42803413de4b8f10abfa97355571a20d55b963b29cce08b18d404e938a3"} Sep 08 08:29:42.916315 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:42.916243 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" podStartSLOduration=13.916217757 podStartE2EDuration="13.916217757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 08:29:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:29:42.912070134 +0000 UTC m=+1196.249709776" watchObservedRunningTime="2026-09-08 08:29:42.916217757 +0000 UTC m=+1196.253857402" Sep 08 08:29:44.950548 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:44.950508 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:44.951051 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:44.950574 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:44.951051 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:44.950593 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:44.951051 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:44.950609 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:44.952921 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:44.952797 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" podUID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.18:6571: connect: connection refused" Sep 08 08:29:44.952921 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:44.952816 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" podUID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.18:8443: connect: connection refused" Sep 08 08:29:44.952921 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:44.952822 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" podUID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.18:8816: connect: connection refused" Sep 08 08:29:44.952921 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:44.952877 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" podUID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerName="online" probeResult="failure" output="Get \"https://10.134.0.18:6567/health\": dial tcp 10.134.0.18:6567: connect: connection refused" Sep 08 08:29:46.569849 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:46.569747 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.17:8443: connect: connection refused" Sep 08 08:29:46.569849 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:46.569772 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.17:6567/health\": dial tcp 10.134.0.17:6567: connect: connection refused" Sep 08 08:29:47.137225 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:47.137188 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2xlvk_e00cae51-f771-4039-81ec-54fa2a48e98f/ovn-acl-logging/0.log" Sep 08 08:29:47.140881 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:47.140860 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2xlvk_e00cae51-f771-4039-81ec-54fa2a48e98f/ovn-acl-logging/0.log" Sep 08 08:29:47.950987 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:47.950942 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" podUID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.18:8443: connect: connection refused" Sep 08 08:29:47.951586 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:47.950939 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" podUID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.18:8816: connect: connection refused" Sep 08 08:29:47.951724 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:47.950939 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" podUID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.18:6571: connect: connection refused" Sep 08 08:29:47.951724 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:47.950989 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" podUID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerName="online" probeResult="failure" output="Get \"https://10.134.0.18:6567/health\": dial tcp 10.134.0.18:6567: connect: connection refused" Sep 08 08:29:49.950193 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:49.950156 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:49.950708 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:49.950228 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:49.950708 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:49.950249 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:49.950708 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:49.950263 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:50.951726 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:50.951700 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:50.952082 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:50.951887 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:50.952082 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:50.952071 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:50.957177 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:50.957154 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:51.935583 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:51.935552 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:51.935885 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:51.935865 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:51.935992 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:51.935918 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:51.938719 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:51.938700 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:29:56.568966 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:56.568924 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.17:8443: connect: connection refused" Sep 08 08:29:56.569371 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:56.568930 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.17:6567/health\": dial tcp 10.134.0.17:6567: connect: connection refused" Sep 08 08:29:56.569371 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:56.569072 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:29:56.569371 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:56.569250 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:29:58.964734 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:58.964698 2585 generic.go:356] "Generic (PLEG): container finished" podID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerID="5810be777a8a77d8dbbc968b1bbafc10ad054fc7af0a25fae6cd093522d578bb" exitCode=137 Sep 08 08:29:58.964734 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:58.964726 2585 generic.go:356] "Generic (PLEG): container finished" podID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerID="393c0681342cf481286d1ee544cd01114d47d103a99bfda8c049c7ffee98989a" exitCode=137 Sep 08 08:29:58.965136 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:58.964767 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" event={"ID":"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d","Type":"ContainerDied","Data":"5810be777a8a77d8dbbc968b1bbafc10ad054fc7af0a25fae6cd093522d578bb"} Sep 08 08:29:58.965136 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:58.964809 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" event={"ID":"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d","Type":"ContainerDied","Data":"393c0681342cf481286d1ee544cd01114d47d103a99bfda8c049c7ffee98989a"} Sep 08 08:29:59.235094 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:59.235071 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:29:59.279951 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:59.279922 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lhlts\" (UniqueName: \"kubernetes.io/projected/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-kube-api-access-lhlts\") pod \"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d\" (UID: \"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d\") " Sep 08 08:29:59.280109 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:59.279979 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-ui-tls\") pod \"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d\" (UID: \"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d\") " Sep 08 08:29:59.280109 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:59.279997 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-registry-tls\") pod \"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d\" (UID: \"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d\") " Sep 08 08:29:59.280109 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:59.280037 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-feast-data\") pod \"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d\" (UID: \"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d\") " Sep 08 08:29:59.280109 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:59.280073 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-online-tls\") pod \"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d\" (UID: \"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d\") " Sep 08 08:29:59.280109 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:59.280089 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-offline-tls\") pod \"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d\" (UID: \"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d\") " Sep 08 08:29:59.280655 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:59.280612 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-feast-data" (OuterVolumeSpecName: "feast-data") pod "bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" (UID: "bd3a0e5c-9c97-4ff1-9c2f-3807405c608d"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 08:29:59.282520 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:59.282490 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-kube-api-access-lhlts" (OuterVolumeSpecName: "kube-api-access-lhlts") pod "bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" (UID: "bd3a0e5c-9c97-4ff1-9c2f-3807405c608d"). InnerVolumeSpecName "kube-api-access-lhlts". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 08:29:59.282627 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:59.282525 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-online-tls" (OuterVolumeSpecName: "online-tls") pod "bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" (UID: "bd3a0e5c-9c97-4ff1-9c2f-3807405c608d"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:29:59.282627 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:59.282608 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" (UID: "bd3a0e5c-9c97-4ff1-9c2f-3807405c608d"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:29:59.282722 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:59.282657 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" (UID: "bd3a0e5c-9c97-4ff1-9c2f-3807405c608d"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:29:59.282722 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:59.282667 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" (UID: "bd3a0e5c-9c97-4ff1-9c2f-3807405c608d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:29:59.380956 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:59.380915 2585 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-feast-data\") on node \"ip-10-0-143-1.ec2.internal\" DevicePath \"\"" Sep 08 08:29:59.380956 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:59.380961 2585 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-online-tls\") on node \"ip-10-0-143-1.ec2.internal\" DevicePath \"\"" Sep 08 08:29:59.381153 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:59.380969 2585 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-offline-tls\") on node \"ip-10-0-143-1.ec2.internal\" DevicePath \"\"" Sep 08 08:29:59.381153 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:59.380977 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lhlts\" (UniqueName: \"kubernetes.io/projected/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-kube-api-access-lhlts\") on node \"ip-10-0-143-1.ec2.internal\" DevicePath \"\"" Sep 08 08:29:59.381153 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:59.380989 2585 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-ui-tls\") on node \"ip-10-0-143-1.ec2.internal\" DevicePath \"\"" Sep 08 08:29:59.381153 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:59.380998 2585 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d-registry-tls\") on node \"ip-10-0-143-1.ec2.internal\" DevicePath \"\"" Sep 08 08:29:59.971711 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:59.971675 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" event={"ID":"bd3a0e5c-9c97-4ff1-9c2f-3807405c608d","Type":"ContainerDied","Data":"0a560c3dccfae42ebd439f92f7b02b76fd2ce6b81a7d61849b98e56aee635571"} Sep 08 08:29:59.971711 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:59.971708 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh" Sep 08 08:29:59.972235 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:59.971729 2585 scope.go:117] "RemoveContainer" containerID="67ef1817a42be560165ba5678402dcff5bbe0d0813646a4de9833c571cb32b0d" Sep 08 08:29:59.980761 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:59.980742 2585 scope.go:117] "RemoveContainer" containerID="5810be777a8a77d8dbbc968b1bbafc10ad054fc7af0a25fae6cd093522d578bb" Sep 08 08:29:59.988121 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:59.988107 2585 scope.go:117] "RemoveContainer" containerID="220aa93fadde761a75fbfce53ba4e645a1df39bf61c754b1ee95b0455c4764ee" Sep 08 08:29:59.993310 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:59.993286 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh"] Sep 08 08:29:59.996414 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:59.996374 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-npczh"] Sep 08 08:29:59.997472 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:29:59.997458 2585 scope.go:117] "RemoveContainer" containerID="393c0681342cf481286d1ee544cd01114d47d103a99bfda8c049c7ffee98989a" Sep 08 08:30:00.006012 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:00.005995 2585 scope.go:117] "RemoveContainer" containerID="2bb1e7b07a09ed9ba824a296d45d177ee4cab512e2dfbd8854a0a769a2c5577e" Sep 08 08:30:00.012860 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:00.012845 2585 scope.go:117] "RemoveContainer" containerID="8b71e7ada1fbcbc62044ffc1fab0777fa8ec3657b55ed440ee9298bb7c60b2ae" Sep 08 08:30:01.174627 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:01.174593 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" path="/var/lib/kubelet/pods/bd3a0e5c-9c97-4ff1-9c2f-3807405c608d/volumes" Sep 08 08:30:54.183047 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.183011 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9"] Sep 08 08:30:54.183565 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.183482 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerName="offline" Sep 08 08:30:54.183565 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.183509 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerName="offline" Sep 08 08:30:54.183565 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.183530 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerName="feast-apply" Sep 08 08:30:54.183565 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.183539 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerName="feast-apply" Sep 08 08:30:54.183565 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.183549 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerName="ui" Sep 08 08:30:54.183565 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.183556 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerName="ui" Sep 08 08:30:54.183867 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.183573 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerName="online" Sep 08 08:30:54.183867 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.183581 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerName="online" Sep 08 08:30:54.183867 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.183601 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerName="feast-init" Sep 08 08:30:54.183867 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.183609 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerName="feast-init" Sep 08 08:30:54.183867 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.183620 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerName="registry" Sep 08 08:30:54.183867 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.183628 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerName="registry" Sep 08 08:30:54.183867 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.183707 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerName="online" Sep 08 08:30:54.183867 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.183729 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerName="offline" Sep 08 08:30:54.183867 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.183739 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerName="registry" Sep 08 08:30:54.183867 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.183750 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd3a0e5c-9c97-4ff1-9c2f-3807405c608d" containerName="ui" Sep 08 08:30:54.188717 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.188696 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" Sep 08 08:30:54.191156 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.191133 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 08 08:30:54.191156 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.191133 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 08 08:30:54.191371 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.191244 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 08 08:30:54.191448 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.191421 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 08 08:30:54.191984 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.191969 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 08 08:30:54.192100 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.192082 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-8m6nr\"" Sep 08 08:30:54.192141 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.192088 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 08 08:30:54.198902 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.198882 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9"] Sep 08 08:30:54.210465 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.210444 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/351c4818-276b-41d8-9832-666051d6c940-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-kd8q9\" (UID: \"351c4818-276b-41d8-9832-666051d6c940\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" Sep 08 08:30:54.210567 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.210473 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/351c4818-276b-41d8-9832-666051d6c940-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-kd8q9\" (UID: \"351c4818-276b-41d8-9832-666051d6c940\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" Sep 08 08:30:54.210567 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.210523 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jfm7\" (UniqueName: \"kubernetes.io/projected/351c4818-276b-41d8-9832-666051d6c940-kube-api-access-9jfm7\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-kd8q9\" (UID: \"351c4818-276b-41d8-9832-666051d6c940\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" Sep 08 08:30:54.210683 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.210621 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/351c4818-276b-41d8-9832-666051d6c940-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-kd8q9\" (UID: \"351c4818-276b-41d8-9832-666051d6c940\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" Sep 08 08:30:54.210683 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.210673 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/351c4818-276b-41d8-9832-666051d6c940-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-kd8q9\" (UID: \"351c4818-276b-41d8-9832-666051d6c940\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" Sep 08 08:30:54.210777 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.210702 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/351c4818-276b-41d8-9832-666051d6c940-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-kd8q9\" (UID: \"351c4818-276b-41d8-9832-666051d6c940\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" Sep 08 08:30:54.311143 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.311114 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jfm7\" (UniqueName: \"kubernetes.io/projected/351c4818-276b-41d8-9832-666051d6c940-kube-api-access-9jfm7\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-kd8q9\" (UID: \"351c4818-276b-41d8-9832-666051d6c940\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" Sep 08 08:30:54.311327 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.311160 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/351c4818-276b-41d8-9832-666051d6c940-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-kd8q9\" (UID: \"351c4818-276b-41d8-9832-666051d6c940\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" Sep 08 08:30:54.311327 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.311289 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/351c4818-276b-41d8-9832-666051d6c940-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-kd8q9\" (UID: \"351c4818-276b-41d8-9832-666051d6c940\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" Sep 08 08:30:54.311327 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.311322 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/351c4818-276b-41d8-9832-666051d6c940-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-kd8q9\" (UID: \"351c4818-276b-41d8-9832-666051d6c940\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" Sep 08 08:30:54.311521 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.311364 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/351c4818-276b-41d8-9832-666051d6c940-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-kd8q9\" (UID: \"351c4818-276b-41d8-9832-666051d6c940\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" Sep 08 08:30:54.311521 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.311408 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/351c4818-276b-41d8-9832-666051d6c940-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-kd8q9\" (UID: \"351c4818-276b-41d8-9832-666051d6c940\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" Sep 08 08:30:54.311805 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.311770 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/351c4818-276b-41d8-9832-666051d6c940-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-kd8q9\" (UID: \"351c4818-276b-41d8-9832-666051d6c940\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" Sep 08 08:30:54.312050 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.312028 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/351c4818-276b-41d8-9832-666051d6c940-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-kd8q9\" (UID: \"351c4818-276b-41d8-9832-666051d6c940\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" Sep 08 08:30:54.313731 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.313709 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/351c4818-276b-41d8-9832-666051d6c940-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-kd8q9\" (UID: \"351c4818-276b-41d8-9832-666051d6c940\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" Sep 08 08:30:54.313833 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.313753 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/351c4818-276b-41d8-9832-666051d6c940-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-kd8q9\" (UID: \"351c4818-276b-41d8-9832-666051d6c940\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" Sep 08 08:30:54.313896 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.313852 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/351c4818-276b-41d8-9832-666051d6c940-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-kd8q9\" (UID: \"351c4818-276b-41d8-9832-666051d6c940\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" Sep 08 08:30:54.319298 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.319279 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jfm7\" (UniqueName: \"kubernetes.io/projected/351c4818-276b-41d8-9832-666051d6c940-kube-api-access-9jfm7\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-kd8q9\" (UID: \"351c4818-276b-41d8-9832-666051d6c940\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" Sep 08 08:30:54.498781 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.498712 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" Sep 08 08:30:54.786810 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:54.786737 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9"] Sep 08 08:30:54.790802 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:30:54.790766 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod351c4818_276b_41d8_9832_666051d6c940.slice/crio-99466ae5d71683227c3466a6e8ff50a781df1e7a7d4f6c546e26a52173469ad4 WatchSource:0}: Error finding container 99466ae5d71683227c3466a6e8ff50a781df1e7a7d4f6c546e26a52173469ad4: Status 404 returned error can't find the container with id 99466ae5d71683227c3466a6e8ff50a781df1e7a7d4f6c546e26a52173469ad4 Sep 08 08:30:55.189577 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:55.189529 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" event={"ID":"351c4818-276b-41d8-9832-666051d6c940","Type":"ContainerStarted","Data":"859260f0b3f9bb909be5cffc2f1f522e5819d2a30cad1b1fe3f8090994bc4ac8"} Sep 08 08:30:55.189577 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:30:55.189580 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" event={"ID":"351c4818-276b-41d8-9832-666051d6c940","Type":"ContainerStarted","Data":"99466ae5d71683227c3466a6e8ff50a781df1e7a7d4f6c546e26a52173469ad4"} Sep 08 08:31:00.212026 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:31:00.211992 2585 generic.go:356] "Generic (PLEG): container finished" podID="351c4818-276b-41d8-9832-666051d6c940" containerID="859260f0b3f9bb909be5cffc2f1f522e5819d2a30cad1b1fe3f8090994bc4ac8" exitCode=0 Sep 08 08:31:00.212362 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:31:00.212061 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" event={"ID":"351c4818-276b-41d8-9832-666051d6c940","Type":"ContainerDied","Data":"859260f0b3f9bb909be5cffc2f1f522e5819d2a30cad1b1fe3f8090994bc4ac8"} Sep 08 08:31:01.217936 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:31:01.217893 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" event={"ID":"351c4818-276b-41d8-9832-666051d6c940","Type":"ContainerStarted","Data":"19dfee894aa5f0691afe16cef26360d3c0391d4260a599fdf7d635c28d78bb7a"} Sep 08 08:31:23.306503 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:31:23.306466 2585 generic.go:356] "Generic (PLEG): container finished" podID="351c4818-276b-41d8-9832-666051d6c940" containerID="19dfee894aa5f0691afe16cef26360d3c0391d4260a599fdf7d635c28d78bb7a" exitCode=0 Sep 08 08:31:23.306900 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:31:23.306544 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" event={"ID":"351c4818-276b-41d8-9832-666051d6c940","Type":"ContainerDied","Data":"19dfee894aa5f0691afe16cef26360d3c0391d4260a599fdf7d635c28d78bb7a"} Sep 08 08:31:24.315922 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:31:24.315883 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" event={"ID":"351c4818-276b-41d8-9832-666051d6c940","Type":"ContainerStarted","Data":"9005c1a4ad72d83e4ad22ed3b14fb6cfb8d6922b58943fb4a116b168d0325481"} Sep 08 08:31:24.316415 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:31:24.315943 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" event={"ID":"351c4818-276b-41d8-9832-666051d6c940","Type":"ContainerStarted","Data":"35ac05517025696e9c972701e875b586dd5439718c18a01df8d5d0e30e241bcb"} Sep 08 08:31:24.316415 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:31:24.315964 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" event={"ID":"351c4818-276b-41d8-9832-666051d6c940","Type":"ContainerStarted","Data":"3af19ebdad0da2194e8224d4639a8173a94eb1150b95b9ab85705189d2f32f7e"} Sep 08 08:31:24.345762 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:31:24.345711 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" podStartSLOduration=30.345693699999998 podStartE2EDuration="30.3456937s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 08:30:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:31:24.343107879 +0000 UTC m=+1297.680747522" watchObservedRunningTime="2026-09-08 08:31:24.3456937 +0000 UTC m=+1297.683333340" Sep 08 08:31:24.499460 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:31:24.499351 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" Sep 08 08:31:24.499460 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:31:24.499427 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" Sep 08 08:31:24.499460 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:31:24.499448 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" Sep 08 08:31:24.499460 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:31:24.499465 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" Sep 08 08:31:24.499831 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:31:24.499481 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" Sep 08 08:31:24.499831 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:31:24.499497 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" Sep 08 08:31:24.501239 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:31:24.501199 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" podUID="351c4818-276b-41d8-9832-666051d6c940" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.19:8443: connect: connection refused" Sep 08 08:31:24.501351 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:31:24.501224 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" podUID="351c4818-276b-41d8-9832-666051d6c940" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.19:8816: connect: connection refused" Sep 08 08:31:24.501351 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:31:24.501256 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" podUID="351c4818-276b-41d8-9832-666051d6c940" containerName="online" probeResult="failure" output="Get \"https://10.134.0.19:6567/health\": dial tcp 10.134.0.19:6567: connect: connection refused" Sep 08 08:31:27.499469 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:31:27.499355 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" podUID="351c4818-276b-41d8-9832-666051d6c940" containerName="online" probeResult="failure" output="Get \"https://10.134.0.19:6567/health\": dial tcp 10.134.0.19:6567: connect: connection refused" Sep 08 08:31:27.499915 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:31:27.499367 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" podUID="351c4818-276b-41d8-9832-666051d6c940" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.19:8816: connect: connection refused" Sep 08 08:31:27.499915 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:31:27.499352 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" podUID="351c4818-276b-41d8-9832-666051d6c940" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.19:8443: connect: connection refused" Sep 08 08:31:30.499192 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:31:30.499148 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" podUID="351c4818-276b-41d8-9832-666051d6c940" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.19:8443: connect: connection refused" Sep 08 08:31:30.499698 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:31:30.499208 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" podUID="351c4818-276b-41d8-9832-666051d6c940" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.19:8816: connect: connection refused" Sep 08 08:31:30.499698 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:31:30.499241 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" podUID="351c4818-276b-41d8-9832-666051d6c940" containerName="online" probeResult="failure" output="Get \"https://10.134.0.19:6567/health\": dial tcp 10.134.0.19:6567: connect: connection refused" Sep 08 08:31:33.500067 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:31:33.500039 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" Sep 08 08:31:33.500457 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:31:33.500156 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" Sep 08 08:31:33.500849 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:31:33.500832 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" Sep 08 08:31:33.501083 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:31:33.501072 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" Sep 08 08:31:34.281071 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:31:34.281036 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" Sep 08 08:31:35.107803 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:31:35.107769 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" Sep 08 08:31:35.503551 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:31:35.503509 2585 prober.go:120] "Probe failed" probeType="Liveness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" podUID="351c4818-276b-41d8-9832-666051d6c940" containerName="online" probeResult="failure" output="Get \"https://10.134.0.19:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 08 08:31:55.503529 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:31:55.503483 2585 prober.go:120] "Probe failed" probeType="Liveness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" podUID="351c4818-276b-41d8-9832-666051d6c940" containerName="online" probeResult="failure" output="Get \"https://10.134.0.19:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 08 08:32:15.502612 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:32:15.502513 2585 prober.go:120] "Probe failed" probeType="Liveness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" podUID="351c4818-276b-41d8-9832-666051d6c940" containerName="online" probeResult="failure" output="Get \"https://10.134.0.19:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 08 08:32:35.376516 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:32:35.376473 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" podUID="351c4818-276b-41d8-9832-666051d6c940" containerName="online" probeResult="failure" output="Get \"https://10.134.0.19:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 08 08:32:35.500400 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:32:35.500338 2585 prober.go:120] "Probe failed" probeType="Liveness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" podUID="351c4818-276b-41d8-9832-666051d6c940" containerName="online" probeResult="failure" output="Get \"https://10.134.0.19:6567/health\": context deadline exceeded" Sep 08 08:32:36.501871 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:32:36.501842 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9"] Sep 08 08:32:36.502288 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:32:36.502046 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" podUID="351c4818-276b-41d8-9832-666051d6c940" containerName="online" containerID="cri-o://3af19ebdad0da2194e8224d4639a8173a94eb1150b95b9ab85705189d2f32f7e" gracePeriod=30 Sep 08 08:32:36.502288 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:32:36.502100 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" podUID="351c4818-276b-41d8-9832-666051d6c940" containerName="ui" containerID="cri-o://9005c1a4ad72d83e4ad22ed3b14fb6cfb8d6922b58943fb4a116b168d0325481" gracePeriod=30 Sep 08 08:32:36.502288 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:32:36.502117 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" podUID="351c4818-276b-41d8-9832-666051d6c940" containerName="offline" containerID="cri-o://35ac05517025696e9c972701e875b586dd5439718c18a01df8d5d0e30e241bcb" gracePeriod=30 Sep 08 08:32:36.653534 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:32:36.653500 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz"] Sep 08 08:32:36.653812 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:32:36.653785 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" podUID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerName="registry" containerID="cri-o://f4a1e42803413de4b8f10abfa97355571a20d55b963b29cce08b18d404e938a3" gracePeriod=30 Sep 08 08:32:36.653906 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:32:36.653807 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" podUID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerName="ui" containerID="cri-o://a9dd3978afa4a917c52353aea7e4b26c1de84ad375b96d288a0fdd2ae3d5f1a0" gracePeriod=30 Sep 08 08:32:36.653906 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:32:36.653838 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" podUID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerName="offline" containerID="cri-o://3fd0179ef3ac841a69341c92512b880cd3dd350c6872a81d35af4db0daea4b73" gracePeriod=30 Sep 08 08:32:36.653906 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:32:36.653859 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" podUID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerName="online" containerID="cri-o://1b3287ee5d75dae31c1dd742e6bd412b4cf299d516bb7c8f77a4c34d2e3abfa2" gracePeriod=30 Sep 08 08:32:37.623712 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:32:37.623682 2585 generic.go:356] "Generic (PLEG): container finished" podID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerID="a9dd3978afa4a917c52353aea7e4b26c1de84ad375b96d288a0fdd2ae3d5f1a0" exitCode=0 Sep 08 08:32:37.623712 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:32:37.623708 2585 generic.go:356] "Generic (PLEG): container finished" podID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerID="1b3287ee5d75dae31c1dd742e6bd412b4cf299d516bb7c8f77a4c34d2e3abfa2" exitCode=0 Sep 08 08:32:37.624125 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:32:37.623760 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" event={"ID":"0e2d324f-2a5f-488a-9bc2-a0a56af7815b","Type":"ContainerDied","Data":"a9dd3978afa4a917c52353aea7e4b26c1de84ad375b96d288a0fdd2ae3d5f1a0"} Sep 08 08:32:37.624125 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:32:37.623816 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" event={"ID":"0e2d324f-2a5f-488a-9bc2-a0a56af7815b","Type":"ContainerDied","Data":"1b3287ee5d75dae31c1dd742e6bd412b4cf299d516bb7c8f77a4c34d2e3abfa2"} Sep 08 08:32:37.626321 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:32:37.626302 2585 generic.go:356] "Generic (PLEG): container finished" podID="351c4818-276b-41d8-9832-666051d6c940" containerID="9005c1a4ad72d83e4ad22ed3b14fb6cfb8d6922b58943fb4a116b168d0325481" exitCode=0 Sep 08 08:32:37.626321 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:32:37.626317 2585 generic.go:356] "Generic (PLEG): container finished" podID="351c4818-276b-41d8-9832-666051d6c940" containerID="3af19ebdad0da2194e8224d4639a8173a94eb1150b95b9ab85705189d2f32f7e" exitCode=0 Sep 08 08:32:37.626488 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:32:37.626346 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" event={"ID":"351c4818-276b-41d8-9832-666051d6c940","Type":"ContainerDied","Data":"9005c1a4ad72d83e4ad22ed3b14fb6cfb8d6922b58943fb4a116b168d0325481"} Sep 08 08:32:37.626488 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:32:37.626371 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" event={"ID":"351c4818-276b-41d8-9832-666051d6c940","Type":"ContainerDied","Data":"3af19ebdad0da2194e8224d4639a8173a94eb1150b95b9ab85705189d2f32f7e"} Sep 08 08:32:41.935322 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:32:41.935279 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" podUID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.18:8443: connect: connection refused" Sep 08 08:32:41.935817 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:32:41.935293 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" podUID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerName="online" probeResult="failure" output="Get \"https://10.134.0.18:6567/health\": dial tcp 10.134.0.18:6567: connect: connection refused" Sep 08 08:32:43.500768 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:32:43.500724 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" podUID="351c4818-276b-41d8-9832-666051d6c940" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.19:8443: connect: connection refused" Sep 08 08:32:44.372301 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:32:44.372258 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" podUID="351c4818-276b-41d8-9832-666051d6c940" containerName="online" probeResult="failure" output="Get \"https://10.134.0.19:6567/health\": dial tcp 10.134.0.19:6567: connect: connection refused" Sep 08 08:32:51.935536 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:32:51.935490 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" podUID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.18:8443: connect: connection refused" Sep 08 08:32:51.935990 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:32:51.935553 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" podUID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerName="online" probeResult="failure" output="Get \"https://10.134.0.18:6567/health\": dial tcp 10.134.0.18:6567: connect: connection refused" Sep 08 08:32:53.500773 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:32:53.500725 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" podUID="351c4818-276b-41d8-9832-666051d6c940" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.19:8443: connect: connection refused" Sep 08 08:32:54.372861 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:32:54.372812 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" podUID="351c4818-276b-41d8-9832-666051d6c940" containerName="online" probeResult="failure" output="Get \"https://10.134.0.19:6567/health\": dial tcp 10.134.0.19:6567: connect: connection refused" Sep 08 08:32:54.373030 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:32:54.372950 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" Sep 08 08:33:01.935806 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:01.935759 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" podUID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.18:8443: connect: connection refused" Sep 08 08:33:01.936365 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:01.935763 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" podUID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerName="online" probeResult="failure" output="Get \"https://10.134.0.18:6567/health\": dial tcp 10.134.0.18:6567: connect: connection refused" Sep 08 08:33:01.936365 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:01.935896 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:33:01.936365 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:01.935960 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:33:03.501070 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:03.501025 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" podUID="351c4818-276b-41d8-9832-666051d6c940" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.19:8443: connect: connection refused" Sep 08 08:33:03.501591 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:03.501181 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" Sep 08 08:33:04.372745 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:04.372705 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" podUID="351c4818-276b-41d8-9832-666051d6c940" containerName="online" probeResult="failure" output="Get \"https://10.134.0.19:6567/health\": dial tcp 10.134.0.19:6567: connect: connection refused" Sep 08 08:33:06.743517 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:06.743488 2585 generic.go:356] "Generic (PLEG): container finished" podID="351c4818-276b-41d8-9832-666051d6c940" containerID="35ac05517025696e9c972701e875b586dd5439718c18a01df8d5d0e30e241bcb" exitCode=137 Sep 08 08:33:06.743918 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:06.743560 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" event={"ID":"351c4818-276b-41d8-9832-666051d6c940","Type":"ContainerDied","Data":"35ac05517025696e9c972701e875b586dd5439718c18a01df8d5d0e30e241bcb"} Sep 08 08:33:06.746642 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:06.746623 2585 generic.go:356] "Generic (PLEG): container finished" podID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerID="3fd0179ef3ac841a69341c92512b880cd3dd350c6872a81d35af4db0daea4b73" exitCode=137 Sep 08 08:33:06.746642 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:06.746639 2585 generic.go:356] "Generic (PLEG): container finished" podID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerID="f4a1e42803413de4b8f10abfa97355571a20d55b963b29cce08b18d404e938a3" exitCode=137 Sep 08 08:33:06.746779 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:06.746692 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" event={"ID":"0e2d324f-2a5f-488a-9bc2-a0a56af7815b","Type":"ContainerDied","Data":"3fd0179ef3ac841a69341c92512b880cd3dd350c6872a81d35af4db0daea4b73"} Sep 08 08:33:06.746779 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:06.746730 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" event={"ID":"0e2d324f-2a5f-488a-9bc2-a0a56af7815b","Type":"ContainerDied","Data":"f4a1e42803413de4b8f10abfa97355571a20d55b963b29cce08b18d404e938a3"} Sep 08 08:33:07.151081 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.151058 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" Sep 08 08:33:07.236783 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.236751 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/351c4818-276b-41d8-9832-666051d6c940-feast-data\") pod \"351c4818-276b-41d8-9832-666051d6c940\" (UID: \"351c4818-276b-41d8-9832-666051d6c940\") " Sep 08 08:33:07.236783 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.236784 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/351c4818-276b-41d8-9832-666051d6c940-offline-tls\") pod \"351c4818-276b-41d8-9832-666051d6c940\" (UID: \"351c4818-276b-41d8-9832-666051d6c940\") " Sep 08 08:33:07.236982 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.236818 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9jfm7\" (UniqueName: \"kubernetes.io/projected/351c4818-276b-41d8-9832-666051d6c940-kube-api-access-9jfm7\") pod \"351c4818-276b-41d8-9832-666051d6c940\" (UID: \"351c4818-276b-41d8-9832-666051d6c940\") " Sep 08 08:33:07.236982 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.236864 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/351c4818-276b-41d8-9832-666051d6c940-online-tls\") pod \"351c4818-276b-41d8-9832-666051d6c940\" (UID: \"351c4818-276b-41d8-9832-666051d6c940\") " Sep 08 08:33:07.237125 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.236999 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/351c4818-276b-41d8-9832-666051d6c940-registry-tls\") pod \"351c4818-276b-41d8-9832-666051d6c940\" (UID: \"351c4818-276b-41d8-9832-666051d6c940\") " Sep 08 08:33:07.237125 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.237081 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/351c4818-276b-41d8-9832-666051d6c940-ui-tls\") pod \"351c4818-276b-41d8-9832-666051d6c940\" (UID: \"351c4818-276b-41d8-9832-666051d6c940\") " Sep 08 08:33:07.237475 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.237367 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/351c4818-276b-41d8-9832-666051d6c940-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "351c4818-276b-41d8-9832-666051d6c940" (UID: "351c4818-276b-41d8-9832-666051d6c940"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 08:33:07.237475 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.237430 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/351c4818-276b-41d8-9832-666051d6c940-feast-data" (OuterVolumeSpecName: "feast-data") pod "351c4818-276b-41d8-9832-666051d6c940" (UID: "351c4818-276b-41d8-9832-666051d6c940"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 08:33:07.239573 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.239534 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/351c4818-276b-41d8-9832-666051d6c940-kube-api-access-9jfm7" (OuterVolumeSpecName: "kube-api-access-9jfm7") pod "351c4818-276b-41d8-9832-666051d6c940" (UID: "351c4818-276b-41d8-9832-666051d6c940"). InnerVolumeSpecName "kube-api-access-9jfm7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 08:33:07.239682 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.239617 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/351c4818-276b-41d8-9832-666051d6c940-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "351c4818-276b-41d8-9832-666051d6c940" (UID: "351c4818-276b-41d8-9832-666051d6c940"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:33:07.239728 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.239678 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/351c4818-276b-41d8-9832-666051d6c940-online-tls" (OuterVolumeSpecName: "online-tls") pod "351c4818-276b-41d8-9832-666051d6c940" (UID: "351c4818-276b-41d8-9832-666051d6c940"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:33:07.239810 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.239792 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/351c4818-276b-41d8-9832-666051d6c940-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "351c4818-276b-41d8-9832-666051d6c940" (UID: "351c4818-276b-41d8-9832-666051d6c940"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:33:07.289643 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.289621 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:33:07.338063 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.338035 2585 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/351c4818-276b-41d8-9832-666051d6c940-feast-data\") on node \"ip-10-0-143-1.ec2.internal\" DevicePath \"\"" Sep 08 08:33:07.338063 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.338057 2585 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/351c4818-276b-41d8-9832-666051d6c940-offline-tls\") on node \"ip-10-0-143-1.ec2.internal\" DevicePath \"\"" Sep 08 08:33:07.338063 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.338066 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jfm7\" (UniqueName: \"kubernetes.io/projected/351c4818-276b-41d8-9832-666051d6c940-kube-api-access-9jfm7\") on node \"ip-10-0-143-1.ec2.internal\" DevicePath \"\"" Sep 08 08:33:07.338239 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.338074 2585 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/351c4818-276b-41d8-9832-666051d6c940-online-tls\") on node \"ip-10-0-143-1.ec2.internal\" DevicePath \"\"" Sep 08 08:33:07.338239 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.338082 2585 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/351c4818-276b-41d8-9832-666051d6c940-registry-tls\") on node \"ip-10-0-143-1.ec2.internal\" DevicePath \"\"" Sep 08 08:33:07.338239 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.338090 2585 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/351c4818-276b-41d8-9832-666051d6c940-ui-tls\") on node \"ip-10-0-143-1.ec2.internal\" DevicePath \"\"" Sep 08 08:33:07.438374 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.438337 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-ui-tls\") pod \"0e2d324f-2a5f-488a-9bc2-a0a56af7815b\" (UID: \"0e2d324f-2a5f-488a-9bc2-a0a56af7815b\") " Sep 08 08:33:07.438583 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.438447 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-feast-data\") pod \"0e2d324f-2a5f-488a-9bc2-a0a56af7815b\" (UID: \"0e2d324f-2a5f-488a-9bc2-a0a56af7815b\") " Sep 08 08:33:07.438583 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.438465 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-offline-tls\") pod \"0e2d324f-2a5f-488a-9bc2-a0a56af7815b\" (UID: \"0e2d324f-2a5f-488a-9bc2-a0a56af7815b\") " Sep 08 08:33:07.438583 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.438484 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-registry-tls\") pod \"0e2d324f-2a5f-488a-9bc2-a0a56af7815b\" (UID: \"0e2d324f-2a5f-488a-9bc2-a0a56af7815b\") " Sep 08 08:33:07.438583 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.438510 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-online-tls\") pod \"0e2d324f-2a5f-488a-9bc2-a0a56af7815b\" (UID: \"0e2d324f-2a5f-488a-9bc2-a0a56af7815b\") " Sep 08 08:33:07.438583 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.438542 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6rmnm\" (UniqueName: \"kubernetes.io/projected/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-kube-api-access-6rmnm\") pod \"0e2d324f-2a5f-488a-9bc2-a0a56af7815b\" (UID: \"0e2d324f-2a5f-488a-9bc2-a0a56af7815b\") " Sep 08 08:33:07.439172 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.439059 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-feast-data" (OuterVolumeSpecName: "feast-data") pod "0e2d324f-2a5f-488a-9bc2-a0a56af7815b" (UID: "0e2d324f-2a5f-488a-9bc2-a0a56af7815b"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 08:33:07.440711 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.440683 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-kube-api-access-6rmnm" (OuterVolumeSpecName: "kube-api-access-6rmnm") pod "0e2d324f-2a5f-488a-9bc2-a0a56af7815b" (UID: "0e2d324f-2a5f-488a-9bc2-a0a56af7815b"). InnerVolumeSpecName "kube-api-access-6rmnm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 08:33:07.440711 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.440690 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "0e2d324f-2a5f-488a-9bc2-a0a56af7815b" (UID: "0e2d324f-2a5f-488a-9bc2-a0a56af7815b"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:33:07.440711 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.440700 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "0e2d324f-2a5f-488a-9bc2-a0a56af7815b" (UID: "0e2d324f-2a5f-488a-9bc2-a0a56af7815b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:33:07.441073 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.441055 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "0e2d324f-2a5f-488a-9bc2-a0a56af7815b" (UID: "0e2d324f-2a5f-488a-9bc2-a0a56af7815b"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:33:07.441126 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.441068 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-online-tls" (OuterVolumeSpecName: "online-tls") pod "0e2d324f-2a5f-488a-9bc2-a0a56af7815b" (UID: "0e2d324f-2a5f-488a-9bc2-a0a56af7815b"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:33:07.539646 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.539586 2585 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-feast-data\") on node \"ip-10-0-143-1.ec2.internal\" DevicePath \"\"" Sep 08 08:33:07.539646 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.539608 2585 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-offline-tls\") on node \"ip-10-0-143-1.ec2.internal\" DevicePath \"\"" Sep 08 08:33:07.539646 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.539616 2585 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-registry-tls\") on node \"ip-10-0-143-1.ec2.internal\" DevicePath \"\"" Sep 08 08:33:07.539646 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.539624 2585 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-online-tls\") on node \"ip-10-0-143-1.ec2.internal\" DevicePath \"\"" Sep 08 08:33:07.539646 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.539633 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rmnm\" (UniqueName: \"kubernetes.io/projected/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-kube-api-access-6rmnm\") on node \"ip-10-0-143-1.ec2.internal\" DevicePath \"\"" Sep 08 08:33:07.539646 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.539641 2585 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0e2d324f-2a5f-488a-9bc2-a0a56af7815b-ui-tls\") on node \"ip-10-0-143-1.ec2.internal\" DevicePath \"\"" Sep 08 08:33:07.753563 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.753539 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" Sep 08 08:33:07.753563 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.753547 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9" event={"ID":"351c4818-276b-41d8-9832-666051d6c940","Type":"ContainerDied","Data":"99466ae5d71683227c3466a6e8ff50a781df1e7a7d4f6c546e26a52173469ad4"} Sep 08 08:33:07.753994 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.753606 2585 scope.go:117] "RemoveContainer" containerID="9005c1a4ad72d83e4ad22ed3b14fb6cfb8d6922b58943fb4a116b168d0325481" Sep 08 08:33:07.756805 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.756773 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" event={"ID":"0e2d324f-2a5f-488a-9bc2-a0a56af7815b","Type":"ContainerDied","Data":"4e4b3c6255edddf3cbfd439c1464201a42b16893397185aeee1a869cb5574cda"} Sep 08 08:33:07.756919 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.756867 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz" Sep 08 08:33:07.763003 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.762978 2585 scope.go:117] "RemoveContainer" containerID="35ac05517025696e9c972701e875b586dd5439718c18a01df8d5d0e30e241bcb" Sep 08 08:33:07.771067 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.771053 2585 scope.go:117] "RemoveContainer" containerID="3af19ebdad0da2194e8224d4639a8173a94eb1150b95b9ab85705189d2f32f7e" Sep 08 08:33:07.778242 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.778225 2585 scope.go:117] "RemoveContainer" containerID="19dfee894aa5f0691afe16cef26360d3c0391d4260a599fdf7d635c28d78bb7a" Sep 08 08:33:07.779011 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.778976 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9"] Sep 08 08:33:07.784629 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.784610 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-kd8q9"] Sep 08 08:33:07.788218 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.788202 2585 scope.go:117] "RemoveContainer" containerID="859260f0b3f9bb909be5cffc2f1f522e5819d2a30cad1b1fe3f8090994bc4ac8" Sep 08 08:33:07.795309 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.795153 2585 scope.go:117] "RemoveContainer" containerID="a9dd3978afa4a917c52353aea7e4b26c1de84ad375b96d288a0fdd2ae3d5f1a0" Sep 08 08:33:07.796657 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.796641 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz"] Sep 08 08:33:07.800718 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.800698 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-fzfvz"] Sep 08 08:33:07.802931 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.802916 2585 scope.go:117] "RemoveContainer" containerID="3fd0179ef3ac841a69341c92512b880cd3dd350c6872a81d35af4db0daea4b73" Sep 08 08:33:07.810186 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.810171 2585 scope.go:117] "RemoveContainer" containerID="1b3287ee5d75dae31c1dd742e6bd412b4cf299d516bb7c8f77a4c34d2e3abfa2" Sep 08 08:33:07.817424 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.817408 2585 scope.go:117] "RemoveContainer" containerID="f4a1e42803413de4b8f10abfa97355571a20d55b963b29cce08b18d404e938a3" Sep 08 08:33:07.826962 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.826947 2585 scope.go:117] "RemoveContainer" containerID="7487bb1aa19fb506d8f6458101a1c000535df37cdb92969b4ec825411203c717" Sep 08 08:33:07.833808 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:07.833793 2585 scope.go:117] "RemoveContainer" containerID="fb83f7178ca832defa28c3cc24e52aa560e3782c48ab8976a92644d2fceb991d" Sep 08 08:33:09.174591 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:09.174562 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" path="/var/lib/kubelet/pods/0e2d324f-2a5f-488a-9bc2-a0a56af7815b/volumes" Sep 08 08:33:09.175132 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:09.175117 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="351c4818-276b-41d8-9832-666051d6c940" path="/var/lib/kubelet/pods/351c4818-276b-41d8-9832-666051d6c940/volumes" Sep 08 08:33:26.197972 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:26.197936 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-fm56j"] Sep 08 08:33:26.198338 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:26.198129 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-fm56j" podUID="1fce6ff0-388f-4c13-9a83-9c7ece9b557b" containerName="manager" containerID="cri-o://1a7fb0dd17a08da1a47c65aa07f1a23697509c7a940e62710a3125399a781375" gracePeriod=10 Sep 08 08:33:26.331471 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:26.331428 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-fm56j" podUID="1fce6ff0-388f-4c13-9a83-9c7ece9b557b" containerName="manager" probeResult="failure" output="Get \"http://10.134.0.16:8081/readyz\": dial tcp 10.134.0.16:8081: connect: connection refused" Sep 08 08:33:26.440007 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:26.439986 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-fm56j" Sep 08 08:33:26.491341 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:26.491269 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qp4zg\" (UniqueName: \"kubernetes.io/projected/1fce6ff0-388f-4c13-9a83-9c7ece9b557b-kube-api-access-qp4zg\") pod \"1fce6ff0-388f-4c13-9a83-9c7ece9b557b\" (UID: \"1fce6ff0-388f-4c13-9a83-9c7ece9b557b\") " Sep 08 08:33:26.493555 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:26.493523 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fce6ff0-388f-4c13-9a83-9c7ece9b557b-kube-api-access-qp4zg" (OuterVolumeSpecName: "kube-api-access-qp4zg") pod "1fce6ff0-388f-4c13-9a83-9c7ece9b557b" (UID: "1fce6ff0-388f-4c13-9a83-9c7ece9b557b"). InnerVolumeSpecName "kube-api-access-qp4zg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 08:33:26.592724 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:26.592695 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qp4zg\" (UniqueName: \"kubernetes.io/projected/1fce6ff0-388f-4c13-9a83-9c7ece9b557b-kube-api-access-qp4zg\") on node \"ip-10-0-143-1.ec2.internal\" DevicePath \"\"" Sep 08 08:33:26.828526 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:26.828441 2585 generic.go:356] "Generic (PLEG): container finished" podID="1fce6ff0-388f-4c13-9a83-9c7ece9b557b" containerID="1a7fb0dd17a08da1a47c65aa07f1a23697509c7a940e62710a3125399a781375" exitCode=0 Sep 08 08:33:26.828526 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:26.828487 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-fm56j" Sep 08 08:33:26.828526 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:26.828491 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-fm56j" event={"ID":"1fce6ff0-388f-4c13-9a83-9c7ece9b557b","Type":"ContainerDied","Data":"1a7fb0dd17a08da1a47c65aa07f1a23697509c7a940e62710a3125399a781375"} Sep 08 08:33:26.828729 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:26.828544 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-fm56j" event={"ID":"1fce6ff0-388f-4c13-9a83-9c7ece9b557b","Type":"ContainerDied","Data":"91741f61aa09d904d2120f56b760a43a3e63bc86967f95e2c2801c4021da5f05"} Sep 08 08:33:26.828729 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:26.828574 2585 scope.go:117] "RemoveContainer" containerID="1a7fb0dd17a08da1a47c65aa07f1a23697509c7a940e62710a3125399a781375" Sep 08 08:33:26.837054 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:26.837036 2585 scope.go:117] "RemoveContainer" containerID="1a7fb0dd17a08da1a47c65aa07f1a23697509c7a940e62710a3125399a781375" Sep 08 08:33:26.837316 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:33:26.837296 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a7fb0dd17a08da1a47c65aa07f1a23697509c7a940e62710a3125399a781375\": container with ID starting with 1a7fb0dd17a08da1a47c65aa07f1a23697509c7a940e62710a3125399a781375 not found: ID does not exist" containerID="1a7fb0dd17a08da1a47c65aa07f1a23697509c7a940e62710a3125399a781375" Sep 08 08:33:26.837366 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:26.837324 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a7fb0dd17a08da1a47c65aa07f1a23697509c7a940e62710a3125399a781375"} err="failed to get container status \"1a7fb0dd17a08da1a47c65aa07f1a23697509c7a940e62710a3125399a781375\": rpc error: code = NotFound desc = could not find container \"1a7fb0dd17a08da1a47c65aa07f1a23697509c7a940e62710a3125399a781375\": container with ID starting with 1a7fb0dd17a08da1a47c65aa07f1a23697509c7a940e62710a3125399a781375 not found: ID does not exist" Sep 08 08:33:26.851525 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:26.851493 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-fm56j"] Sep 08 08:33:26.852574 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:26.852557 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-fm56j"] Sep 08 08:33:27.175123 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:27.175090 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fce6ff0-388f-4c13-9a83-9c7ece9b557b" path="/var/lib/kubelet/pods/1fce6ff0-388f-4c13-9a83-9c7ece9b557b/volumes" Sep 08 08:33:52.875079 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.875044 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8"] Sep 08 08:33:52.875616 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.875370 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerName="offline" Sep 08 08:33:52.875616 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.875397 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerName="offline" Sep 08 08:33:52.875616 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.875407 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="351c4818-276b-41d8-9832-666051d6c940" containerName="ui" Sep 08 08:33:52.875616 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.875412 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="351c4818-276b-41d8-9832-666051d6c940" containerName="ui" Sep 08 08:33:52.875616 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.875423 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerName="online" Sep 08 08:33:52.875616 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.875429 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerName="online" Sep 08 08:33:52.875616 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.875433 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="351c4818-276b-41d8-9832-666051d6c940" containerName="online" Sep 08 08:33:52.875616 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.875438 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="351c4818-276b-41d8-9832-666051d6c940" containerName="online" Sep 08 08:33:52.875616 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.875447 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerName="feast-apply" Sep 08 08:33:52.875616 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.875451 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerName="feast-apply" Sep 08 08:33:52.875616 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.875459 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerName="registry" Sep 08 08:33:52.875616 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.875464 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerName="registry" Sep 08 08:33:52.875616 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.875472 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="351c4818-276b-41d8-9832-666051d6c940" containerName="feast-apply" Sep 08 08:33:52.875616 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.875476 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="351c4818-276b-41d8-9832-666051d6c940" containerName="feast-apply" Sep 08 08:33:52.875616 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.875482 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerName="feast-init" Sep 08 08:33:52.875616 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.875487 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerName="feast-init" Sep 08 08:33:52.875616 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.875494 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="351c4818-276b-41d8-9832-666051d6c940" containerName="feast-init" Sep 08 08:33:52.875616 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.875498 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="351c4818-276b-41d8-9832-666051d6c940" containerName="feast-init" Sep 08 08:33:52.875616 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.875503 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="351c4818-276b-41d8-9832-666051d6c940" containerName="offline" Sep 08 08:33:52.875616 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.875507 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="351c4818-276b-41d8-9832-666051d6c940" containerName="offline" Sep 08 08:33:52.875616 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.875514 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fce6ff0-388f-4c13-9a83-9c7ece9b557b" containerName="manager" Sep 08 08:33:52.875616 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.875519 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fce6ff0-388f-4c13-9a83-9c7ece9b557b" containerName="manager" Sep 08 08:33:52.875616 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.875524 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerName="ui" Sep 08 08:33:52.875616 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.875528 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerName="ui" Sep 08 08:33:52.875616 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.875569 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerName="registry" Sep 08 08:33:52.875616 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.875578 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fce6ff0-388f-4c13-9a83-9c7ece9b557b" containerName="manager" Sep 08 08:33:52.875616 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.875584 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="351c4818-276b-41d8-9832-666051d6c940" containerName="online" Sep 08 08:33:52.875616 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.875589 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerName="online" Sep 08 08:33:52.875616 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.875595 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="351c4818-276b-41d8-9832-666051d6c940" containerName="offline" Sep 08 08:33:52.875616 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.875601 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="351c4818-276b-41d8-9832-666051d6c940" containerName="ui" Sep 08 08:33:52.875616 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.875607 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerName="offline" Sep 08 08:33:52.875616 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.875612 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e2d324f-2a5f-488a-9bc2-a0a56af7815b" containerName="ui" Sep 08 08:33:52.880597 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.880580 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:33:52.883560 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.883472 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 08 08:33:52.883560 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.883496 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 08 08:33:52.883560 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.883522 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 08 08:33:52.883560 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.883531 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 08 08:33:52.883560 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.883532 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 08 08:33:52.883853 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.883525 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 08 08:33:52.883853 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.883559 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-t6r4t\"" Sep 08 08:33:52.894316 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:52.894297 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8"] Sep 08 08:33:53.004462 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:53.004426 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-qc9f8\" (UID: \"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:33:53.004462 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:53.004460 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-qc9f8\" (UID: \"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:33:53.004720 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:53.004483 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-qc9f8\" (UID: \"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:33:53.004720 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:53.004547 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2d2r\" (UniqueName: \"kubernetes.io/projected/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-kube-api-access-k2d2r\") pod \"feast-simple-feast-setup-57576b8fcf-qc9f8\" (UID: \"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:33:53.004720 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:53.004606 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-qc9f8\" (UID: \"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:33:53.004720 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:53.004658 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-qc9f8\" (UID: \"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:33:53.105240 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:53.105211 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-qc9f8\" (UID: \"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:33:53.105425 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:53.105274 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-qc9f8\" (UID: \"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:33:53.105425 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:53.105301 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-qc9f8\" (UID: \"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:33:53.105425 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:53.105322 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-qc9f8\" (UID: \"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:33:53.105425 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:53.105340 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2d2r\" (UniqueName: \"kubernetes.io/projected/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-kube-api-access-k2d2r\") pod \"feast-simple-feast-setup-57576b8fcf-qc9f8\" (UID: \"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:33:53.105425 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:53.105374 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-qc9f8\" (UID: \"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:33:53.105715 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:33:53.105459 2585 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Sep 08 08:33:53.105715 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:33:53.105480 2585 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Sep 08 08:33:53.105715 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:33:53.105541 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-ui-tls podName:9808c718-f0c0-45d5-ba6e-2ea76bd56d9e nodeName:}" failed. No retries permitted until 2026-09-08 08:33:53.605518929 +0000 UTC m=+1446.943158553 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-ui-tls") pod "feast-simple-feast-setup-57576b8fcf-qc9f8" (UID: "9808c718-f0c0-45d5-ba6e-2ea76bd56d9e") : secret "feast-simple-feast-setup-ui-tls" not found Sep 08 08:33:53.105715 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:33:53.105555 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-offline-tls podName:9808c718-f0c0-45d5-ba6e-2ea76bd56d9e nodeName:}" failed. No retries permitted until 2026-09-08 08:33:53.605548531 +0000 UTC m=+1446.943188149 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-offline-tls") pod "feast-simple-feast-setup-57576b8fcf-qc9f8" (UID: "9808c718-f0c0-45d5-ba6e-2ea76bd56d9e") : secret "feast-simple-feast-setup-offline-tls" not found Sep 08 08:33:53.105883 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:53.105751 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-qc9f8\" (UID: \"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:33:53.107766 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:53.107747 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-qc9f8\" (UID: \"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:33:53.107881 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:53.107801 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-qc9f8\" (UID: \"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:33:53.120019 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:53.119993 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2d2r\" (UniqueName: \"kubernetes.io/projected/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-kube-api-access-k2d2r\") pod \"feast-simple-feast-setup-57576b8fcf-qc9f8\" (UID: \"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:33:53.610365 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:53.610328 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-qc9f8\" (UID: \"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:33:53.610556 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:53.610399 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-qc9f8\" (UID: \"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:33:53.612928 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:53.612902 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-qc9f8\" (UID: \"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:33:53.612928 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:53.612926 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-qc9f8\" (UID: \"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:33:53.805925 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:53.805894 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:33:54.088966 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:54.088942 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8"] Sep 08 08:33:54.091330 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:33:54.091311 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9808c718_f0c0_45d5_ba6e_2ea76bd56d9e.slice/crio-a747d6760067ef2458dc72a7e1e9c760e5107034f760dc1729d50cc2af8392ee WatchSource:0}: Error finding container a747d6760067ef2458dc72a7e1e9c760e5107034f760dc1729d50cc2af8392ee: Status 404 returned error can't find the container with id a747d6760067ef2458dc72a7e1e9c760e5107034f760dc1729d50cc2af8392ee Sep 08 08:33:54.935831 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:54.935793 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" event={"ID":"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e","Type":"ContainerStarted","Data":"666864c4c035a120891bb654a7af253eba15dd280bf152f936d95f54df0b4518"} Sep 08 08:33:54.936001 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:54.935841 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" event={"ID":"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e","Type":"ContainerStarted","Data":"a747d6760067ef2458dc72a7e1e9c760e5107034f760dc1729d50cc2af8392ee"} Sep 08 08:33:59.955563 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:59.955526 2585 generic.go:356] "Generic (PLEG): container finished" podID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerID="666864c4c035a120891bb654a7af253eba15dd280bf152f936d95f54df0b4518" exitCode=0 Sep 08 08:33:59.955951 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:33:59.955585 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" event={"ID":"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e","Type":"ContainerDied","Data":"666864c4c035a120891bb654a7af253eba15dd280bf152f936d95f54df0b4518"} Sep 08 08:34:00.961164 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:34:00.961130 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" event={"ID":"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e","Type":"ContainerStarted","Data":"a93c1140b5518c3d4d11004b99f0c104d0beb40f2a4dd901bd9b4a0ca5e93660"} Sep 08 08:34:05.982159 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:34:05.982121 2585 generic.go:356] "Generic (PLEG): container finished" podID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerID="a93c1140b5518c3d4d11004b99f0c104d0beb40f2a4dd901bd9b4a0ca5e93660" exitCode=0 Sep 08 08:34:05.982556 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:34:05.982189 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" event={"ID":"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e","Type":"ContainerDied","Data":"a93c1140b5518c3d4d11004b99f0c104d0beb40f2a4dd901bd9b4a0ca5e93660"} Sep 08 08:34:06.992307 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:34:06.992253 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" event={"ID":"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e","Type":"ContainerStarted","Data":"e0c5e841ecdacf75e6436e6167fae7237177bff2b0433120d96be6dd3e7058cc"} Sep 08 08:34:06.992804 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:34:06.992337 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" event={"ID":"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e","Type":"ContainerStarted","Data":"433808685ab05362fb434a850fc9506320444c41481679ee3e002763400df5ed"} Sep 08 08:34:06.992804 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:34:06.992360 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" event={"ID":"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e","Type":"ContainerStarted","Data":"2f7762d456d4dbac7514a0deecacb3851851d758df97f9ce288f7836d2555203"} Sep 08 08:34:06.992804 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:34:06.992400 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" event={"ID":"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e","Type":"ContainerStarted","Data":"56cf5183f28f167e2c5cfb0445ba0ad4c1a8f135e786d52995859eb203137ff1"} Sep 08 08:34:07.022078 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:34:07.022016 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" podStartSLOduration=15.021994045 podStartE2EDuration="15.021994045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 08:33:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:34:07.021759708 +0000 UTC m=+1460.359399349" watchObservedRunningTime="2026-09-08 08:34:07.021994045 +0000 UTC m=+1460.359633687" Sep 08 08:34:08.806603 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:34:08.806538 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:34:08.806603 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:34:08.806592 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:34:08.806603 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:34:08.806611 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:34:08.807181 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:34:08.806628 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:34:08.808874 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:34:08.808828 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" podUID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.20:8443: connect: connection refused" Sep 08 08:34:08.809018 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:34:08.808863 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" podUID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.20:6567/health\": dial tcp 10.134.0.20:6567: connect: connection refused" Sep 08 08:34:08.809018 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:34:08.808906 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" podUID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.20:8816: connect: connection refused" Sep 08 08:34:08.809018 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:34:08.808850 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" podUID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.20:6571: connect: connection refused" Sep 08 08:34:11.806732 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:34:11.806683 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" podUID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.20:6571: connect: connection refused" Sep 08 08:34:11.807327 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:34:11.806692 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" podUID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.20:8443: connect: connection refused" Sep 08 08:34:11.807549 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:34:11.806710 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" podUID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.20:8816: connect: connection refused" Sep 08 08:34:11.807670 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:34:11.806750 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" podUID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.20:6567/health\": dial tcp 10.134.0.20:6567: connect: connection refused" Sep 08 08:34:13.806349 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:34:13.806309 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:34:13.806723 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:34:13.806366 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:34:13.806723 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:34:13.806401 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:34:13.806723 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:34:13.806419 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:34:14.806987 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:34:14.806952 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:34:14.807478 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:34:14.807456 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:34:14.807650 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:34:14.807637 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:34:14.807775 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:34:14.807757 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:34:14.812337 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:34:14.812320 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:34:15.035593 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:34:15.035565 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:34:15.035786 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:34:15.035770 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:34:15.038366 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:34:15.038347 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:34:47.167276 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:34:47.167249 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2xlvk_e00cae51-f771-4039-81ec-54fa2a48e98f/ovn-acl-logging/0.log" Sep 08 08:34:47.173029 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:34:47.173001 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2xlvk_e00cae51-f771-4039-81ec-54fa2a48e98f/ovn-acl-logging/0.log" Sep 08 08:35:16.544269 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:16.544191 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8"] Sep 08 08:35:16.544675 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:16.544458 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" podUID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerName="registry" containerID="cri-o://56cf5183f28f167e2c5cfb0445ba0ad4c1a8f135e786d52995859eb203137ff1" gracePeriod=30 Sep 08 08:35:16.544675 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:16.544626 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" podUID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerName="online" containerID="cri-o://2f7762d456d4dbac7514a0deecacb3851851d758df97f9ce288f7836d2555203" gracePeriod=30 Sep 08 08:35:16.544805 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:16.544638 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" podUID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerName="ui" containerID="cri-o://e0c5e841ecdacf75e6436e6167fae7237177bff2b0433120d96be6dd3e7058cc" gracePeriod=30 Sep 08 08:35:16.544805 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:16.544708 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" podUID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerName="offline" containerID="cri-o://433808685ab05362fb434a850fc9506320444c41481679ee3e002763400df5ed" gracePeriod=30 Sep 08 08:35:16.971853 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:16.971818 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m"] Sep 08 08:35:16.975490 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:16.975473 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:16.977961 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:16.977940 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-klxrr\"" Sep 08 08:35:16.991796 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:16.991772 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m"] Sep 08 08:35:17.020708 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:17.020664 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f6bf283b-17f0-49a1-9782-96af409728d1-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nfx5m\" (UID: \"f6bf283b-17f0-49a1-9782-96af409728d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:17.020870 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:17.020723 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f6bf283b-17f0-49a1-9782-96af409728d1-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nfx5m\" (UID: \"f6bf283b-17f0-49a1-9782-96af409728d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:17.020870 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:17.020764 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f6bf283b-17f0-49a1-9782-96af409728d1-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nfx5m\" (UID: \"f6bf283b-17f0-49a1-9782-96af409728d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:17.020870 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:17.020783 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f6bf283b-17f0-49a1-9782-96af409728d1-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nfx5m\" (UID: \"f6bf283b-17f0-49a1-9782-96af409728d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:17.020973 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:17.020884 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82r7z\" (UniqueName: \"kubernetes.io/projected/f6bf283b-17f0-49a1-9782-96af409728d1-kube-api-access-82r7z\") pod \"feast-simple-feast-setup-57576b8fcf-nfx5m\" (UID: \"f6bf283b-17f0-49a1-9782-96af409728d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:17.020973 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:17.020951 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f6bf283b-17f0-49a1-9782-96af409728d1-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-nfx5m\" (UID: \"f6bf283b-17f0-49a1-9782-96af409728d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:17.122559 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:17.122517 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f6bf283b-17f0-49a1-9782-96af409728d1-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nfx5m\" (UID: \"f6bf283b-17f0-49a1-9782-96af409728d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:17.122752 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:17.122574 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f6bf283b-17f0-49a1-9782-96af409728d1-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nfx5m\" (UID: \"f6bf283b-17f0-49a1-9782-96af409728d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:17.122752 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:17.122616 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f6bf283b-17f0-49a1-9782-96af409728d1-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nfx5m\" (UID: \"f6bf283b-17f0-49a1-9782-96af409728d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:17.122752 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:17.122639 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f6bf283b-17f0-49a1-9782-96af409728d1-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nfx5m\" (UID: \"f6bf283b-17f0-49a1-9782-96af409728d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:17.122752 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:35:17.122668 2585 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Sep 08 08:35:17.122752 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:35:17.122705 2585 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found Sep 08 08:35:17.122752 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:35:17.122746 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6bf283b-17f0-49a1-9782-96af409728d1-offline-tls podName:f6bf283b-17f0-49a1-9782-96af409728d1 nodeName:}" failed. No retries permitted until 2026-09-08 08:35:17.622724382 +0000 UTC m=+1530.960364021 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/f6bf283b-17f0-49a1-9782-96af409728d1-offline-tls") pod "feast-simple-feast-setup-57576b8fcf-nfx5m" (UID: "f6bf283b-17f0-49a1-9782-96af409728d1") : secret "feast-simple-feast-setup-offline-tls" not found Sep 08 08:35:17.122752 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:35:17.122745 2585 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Sep 08 08:35:17.123089 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:35:17.122764 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6bf283b-17f0-49a1-9782-96af409728d1-online-tls podName:f6bf283b-17f0-49a1-9782-96af409728d1 nodeName:}" failed. No retries permitted until 2026-09-08 08:35:17.62275603 +0000 UTC m=+1530.960395648 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/f6bf283b-17f0-49a1-9782-96af409728d1-online-tls") pod "feast-simple-feast-setup-57576b8fcf-nfx5m" (UID: "f6bf283b-17f0-49a1-9782-96af409728d1") : secret "feast-simple-feast-setup-online-tls" not found Sep 08 08:35:17.123089 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:17.122779 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82r7z\" (UniqueName: \"kubernetes.io/projected/f6bf283b-17f0-49a1-9782-96af409728d1-kube-api-access-82r7z\") pod \"feast-simple-feast-setup-57576b8fcf-nfx5m\" (UID: \"f6bf283b-17f0-49a1-9782-96af409728d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:17.123089 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:35:17.122746 2585 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Sep 08 08:35:17.123089 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:35:17.122800 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6bf283b-17f0-49a1-9782-96af409728d1-ui-tls podName:f6bf283b-17f0-49a1-9782-96af409728d1 nodeName:}" failed. No retries permitted until 2026-09-08 08:35:17.622784301 +0000 UTC m=+1530.960423921 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/f6bf283b-17f0-49a1-9782-96af409728d1-ui-tls") pod "feast-simple-feast-setup-57576b8fcf-nfx5m" (UID: "f6bf283b-17f0-49a1-9782-96af409728d1") : secret "feast-simple-feast-setup-ui-tls" not found Sep 08 08:35:17.123089 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:17.122840 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f6bf283b-17f0-49a1-9782-96af409728d1-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-nfx5m\" (UID: \"f6bf283b-17f0-49a1-9782-96af409728d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:17.123089 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:35:17.122861 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6bf283b-17f0-49a1-9782-96af409728d1-registry-tls podName:f6bf283b-17f0-49a1-9782-96af409728d1 nodeName:}" failed. No retries permitted until 2026-09-08 08:35:17.622840894 +0000 UTC m=+1530.960480546 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/f6bf283b-17f0-49a1-9782-96af409728d1-registry-tls") pod "feast-simple-feast-setup-57576b8fcf-nfx5m" (UID: "f6bf283b-17f0-49a1-9782-96af409728d1") : secret "feast-simple-feast-setup-registry-tls" not found Sep 08 08:35:17.123283 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:17.123188 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f6bf283b-17f0-49a1-9782-96af409728d1-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-nfx5m\" (UID: \"f6bf283b-17f0-49a1-9782-96af409728d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:17.141211 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:17.141182 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82r7z\" (UniqueName: \"kubernetes.io/projected/f6bf283b-17f0-49a1-9782-96af409728d1-kube-api-access-82r7z\") pod \"feast-simple-feast-setup-57576b8fcf-nfx5m\" (UID: \"f6bf283b-17f0-49a1-9782-96af409728d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:17.279776 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:17.279739 2585 generic.go:356] "Generic (PLEG): container finished" podID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerID="e0c5e841ecdacf75e6436e6167fae7237177bff2b0433120d96be6dd3e7058cc" exitCode=0 Sep 08 08:35:17.279956 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:17.279779 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" event={"ID":"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e","Type":"ContainerDied","Data":"e0c5e841ecdacf75e6436e6167fae7237177bff2b0433120d96be6dd3e7058cc"} Sep 08 08:35:17.626950 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:17.626869 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f6bf283b-17f0-49a1-9782-96af409728d1-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nfx5m\" (UID: \"f6bf283b-17f0-49a1-9782-96af409728d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:17.626950 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:17.626906 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f6bf283b-17f0-49a1-9782-96af409728d1-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nfx5m\" (UID: \"f6bf283b-17f0-49a1-9782-96af409728d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:17.626950 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:17.626930 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f6bf283b-17f0-49a1-9782-96af409728d1-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nfx5m\" (UID: \"f6bf283b-17f0-49a1-9782-96af409728d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:17.626950 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:17.626946 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f6bf283b-17f0-49a1-9782-96af409728d1-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nfx5m\" (UID: \"f6bf283b-17f0-49a1-9782-96af409728d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:17.629459 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:17.629437 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f6bf283b-17f0-49a1-9782-96af409728d1-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nfx5m\" (UID: \"f6bf283b-17f0-49a1-9782-96af409728d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:17.629575 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:17.629487 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f6bf283b-17f0-49a1-9782-96af409728d1-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nfx5m\" (UID: \"f6bf283b-17f0-49a1-9782-96af409728d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:17.629934 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:17.629916 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f6bf283b-17f0-49a1-9782-96af409728d1-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nfx5m\" (UID: \"f6bf283b-17f0-49a1-9782-96af409728d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:17.629974 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:17.629928 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f6bf283b-17f0-49a1-9782-96af409728d1-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nfx5m\" (UID: \"f6bf283b-17f0-49a1-9782-96af409728d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:17.886703 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:17.886675 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:18.167561 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:18.167330 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m"] Sep 08 08:35:18.170177 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:35:18.170150 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6bf283b_17f0_49a1_9782_96af409728d1.slice/crio-713f3779472a9aa5e98894920760a9848305a09d615ad9e15c762bc0242f4d2a WatchSource:0}: Error finding container 713f3779472a9aa5e98894920760a9848305a09d615ad9e15c762bc0242f4d2a: Status 404 returned error can't find the container with id 713f3779472a9aa5e98894920760a9848305a09d615ad9e15c762bc0242f4d2a Sep 08 08:35:18.286815 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:18.286779 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" event={"ID":"f6bf283b-17f0-49a1-9782-96af409728d1","Type":"ContainerStarted","Data":"2b05ae10a934fdd70b2200be8d026f780ca596edadb72eac94174577b270a1a3"} Sep 08 08:35:18.286985 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:18.286834 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" event={"ID":"f6bf283b-17f0-49a1-9782-96af409728d1","Type":"ContainerStarted","Data":"713f3779472a9aa5e98894920760a9848305a09d615ad9e15c762bc0242f4d2a"} Sep 08 08:35:18.289993 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:18.289958 2585 generic.go:356] "Generic (PLEG): container finished" podID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerID="2f7762d456d4dbac7514a0deecacb3851851d758df97f9ce288f7836d2555203" exitCode=0 Sep 08 08:35:18.290119 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:18.290033 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" event={"ID":"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e","Type":"ContainerDied","Data":"2f7762d456d4dbac7514a0deecacb3851851d758df97f9ce288f7836d2555203"} Sep 08 08:35:23.309323 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:23.309290 2585 generic.go:356] "Generic (PLEG): container finished" podID="f6bf283b-17f0-49a1-9782-96af409728d1" containerID="2b05ae10a934fdd70b2200be8d026f780ca596edadb72eac94174577b270a1a3" exitCode=0 Sep 08 08:35:23.309719 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:23.309366 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" event={"ID":"f6bf283b-17f0-49a1-9782-96af409728d1","Type":"ContainerDied","Data":"2b05ae10a934fdd70b2200be8d026f780ca596edadb72eac94174577b270a1a3"} Sep 08 08:35:24.314883 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:24.314852 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" event={"ID":"f6bf283b-17f0-49a1-9782-96af409728d1","Type":"ContainerStarted","Data":"8b8bee39da240da92127eeff280f9f0725af6c62513ee00fa4254315e936aa76"} Sep 08 08:35:25.035928 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:25.035883 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" podUID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.20:6567/health\": dial tcp 10.134.0.20:6567: connect: connection refused" Sep 08 08:35:25.036117 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:25.035879 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" podUID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.20:8443: connect: connection refused" Sep 08 08:35:29.336020 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:29.335987 2585 generic.go:356] "Generic (PLEG): container finished" podID="f6bf283b-17f0-49a1-9782-96af409728d1" containerID="8b8bee39da240da92127eeff280f9f0725af6c62513ee00fa4254315e936aa76" exitCode=0 Sep 08 08:35:29.336421 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:29.336054 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" event={"ID":"f6bf283b-17f0-49a1-9782-96af409728d1","Type":"ContainerDied","Data":"8b8bee39da240da92127eeff280f9f0725af6c62513ee00fa4254315e936aa76"} Sep 08 08:35:30.346166 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:30.346114 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" event={"ID":"f6bf283b-17f0-49a1-9782-96af409728d1","Type":"ContainerStarted","Data":"e75bc372fe53bdb19d4d8b76e85bdb530e4c09c00bb78f27d8ede50a23daca8c"} Sep 08 08:35:30.346673 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:30.346207 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" event={"ID":"f6bf283b-17f0-49a1-9782-96af409728d1","Type":"ContainerStarted","Data":"f15fe4dc98a7ba988a917adb6c4858ad02431aacc387c22031d80275abc08c99"} Sep 08 08:35:30.346673 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:30.346233 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" event={"ID":"f6bf283b-17f0-49a1-9782-96af409728d1","Type":"ContainerStarted","Data":"c725a2ad4e2803369f399ba7c3f6338fe9db60a09978c49f21b76f0a66dfc9cd"} Sep 08 08:35:30.346673 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:30.346252 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" event={"ID":"f6bf283b-17f0-49a1-9782-96af409728d1","Type":"ContainerStarted","Data":"0db7f8cecb90169d39a71cb14fb0708a4d20ac3153583c8bc2573ba78eb23978"} Sep 08 08:35:30.372851 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:30.372783 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" podStartSLOduration=14.372760822 podStartE2EDuration="14.372760822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 08:35:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:35:30.371662677 +0000 UTC m=+1543.709302320" watchObservedRunningTime="2026-09-08 08:35:30.372760822 +0000 UTC m=+1543.710400464" Sep 08 08:35:32.887274 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:32.887229 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:32.887794 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:32.887309 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:32.887794 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:32.887766 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:32.888607 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:32.888043 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:32.889362 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:32.889327 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" podUID="f6bf283b-17f0-49a1-9782-96af409728d1" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.21:8443: connect: connection refused" Sep 08 08:35:32.889496 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:32.889362 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" podUID="f6bf283b-17f0-49a1-9782-96af409728d1" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.21:6571: connect: connection refused" Sep 08 08:35:32.889496 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:32.889478 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" podUID="f6bf283b-17f0-49a1-9782-96af409728d1" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.21:8816: connect: connection refused" Sep 08 08:35:32.889630 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:32.889573 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" podUID="f6bf283b-17f0-49a1-9782-96af409728d1" containerName="online" probeResult="failure" output="Get \"https://10.134.0.21:6567/health\": dial tcp 10.134.0.21:6567: connect: connection refused" Sep 08 08:35:35.035805 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:35.035752 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" podUID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.20:8443: connect: connection refused" Sep 08 08:35:35.036283 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:35.035767 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" podUID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.20:6567/health\": dial tcp 10.134.0.20:6567: connect: connection refused" Sep 08 08:35:35.887555 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:35.887510 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" podUID="f6bf283b-17f0-49a1-9782-96af409728d1" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.21:6571: connect: connection refused" Sep 08 08:35:35.887794 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:35.887556 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" podUID="f6bf283b-17f0-49a1-9782-96af409728d1" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.21:8816: connect: connection refused" Sep 08 08:35:35.887992 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:35.887965 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" podUID="f6bf283b-17f0-49a1-9782-96af409728d1" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.21:8443: connect: connection refused" Sep 08 08:35:35.887992 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:35.887983 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" podUID="f6bf283b-17f0-49a1-9782-96af409728d1" containerName="online" probeResult="failure" output="Get \"https://10.134.0.21:6567/health\": dial tcp 10.134.0.21:6567: connect: connection refused" Sep 08 08:35:37.887179 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:37.887134 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:37.887573 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:37.887192 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:37.887573 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:37.887206 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:37.887573 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:37.887216 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:38.888268 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:38.888237 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:38.888664 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:38.888359 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:38.888664 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:38.888528 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:38.889062 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:38.889041 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:38.889173 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:38.889096 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:38.889173 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:38.889129 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:38.893463 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:38.893444 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:39.400262 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:39.400228 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:35:45.035839 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:45.035792 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" podUID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.20:8443: connect: connection refused" Sep 08 08:35:45.035839 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:45.035820 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" podUID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.20:6567/health\": dial tcp 10.134.0.20:6567: connect: connection refused" Sep 08 08:35:45.036278 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:45.035941 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:35:45.036278 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:45.036014 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:35:47.182769 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.182740 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:35:47.195650 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.195627 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-online-tls\") pod \"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e\" (UID: \"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e\") " Sep 08 08:35:47.195773 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.195694 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2d2r\" (UniqueName: \"kubernetes.io/projected/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-kube-api-access-k2d2r\") pod \"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e\" (UID: \"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e\") " Sep 08 08:35:47.195773 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.195741 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-feast-data\") pod \"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e\" (UID: \"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e\") " Sep 08 08:35:47.195773 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.195764 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-ui-tls\") pod \"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e\" (UID: \"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e\") " Sep 08 08:35:47.195920 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.195887 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-registry-tls\") pod \"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e\" (UID: \"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e\") " Sep 08 08:35:47.195972 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.195956 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-offline-tls\") pod \"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e\" (UID: \"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e\") " Sep 08 08:35:47.196284 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.196252 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-feast-data" (OuterVolumeSpecName: "feast-data") pod "9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" (UID: "9808c718-f0c0-45d5-ba6e-2ea76bd56d9e"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 08:35:47.198311 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.198290 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" (UID: "9808c718-f0c0-45d5-ba6e-2ea76bd56d9e"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:35:47.198480 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.198454 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-kube-api-access-k2d2r" (OuterVolumeSpecName: "kube-api-access-k2d2r") pod "9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" (UID: "9808c718-f0c0-45d5-ba6e-2ea76bd56d9e"). InnerVolumeSpecName "kube-api-access-k2d2r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 08:35:47.198558 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.198473 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-online-tls" (OuterVolumeSpecName: "online-tls") pod "9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" (UID: "9808c718-f0c0-45d5-ba6e-2ea76bd56d9e"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:35:47.198558 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.198503 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" (UID: "9808c718-f0c0-45d5-ba6e-2ea76bd56d9e"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:35:47.198558 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.198520 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" (UID: "9808c718-f0c0-45d5-ba6e-2ea76bd56d9e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:35:47.297162 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.297133 2585 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-feast-data\") on node \"ip-10-0-143-1.ec2.internal\" DevicePath \"\"" Sep 08 08:35:47.297162 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.297158 2585 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-ui-tls\") on node \"ip-10-0-143-1.ec2.internal\" DevicePath \"\"" Sep 08 08:35:47.297162 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.297166 2585 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-registry-tls\") on node \"ip-10-0-143-1.ec2.internal\" DevicePath \"\"" Sep 08 08:35:47.297370 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.297173 2585 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-offline-tls\") on node \"ip-10-0-143-1.ec2.internal\" DevicePath \"\"" Sep 08 08:35:47.297370 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.297182 2585 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-online-tls\") on node \"ip-10-0-143-1.ec2.internal\" DevicePath \"\"" Sep 08 08:35:47.297370 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.297191 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2d2r\" (UniqueName: \"kubernetes.io/projected/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e-kube-api-access-k2d2r\") on node \"ip-10-0-143-1.ec2.internal\" DevicePath \"\"" Sep 08 08:35:47.430901 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.430866 2585 generic.go:356] "Generic (PLEG): container finished" podID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerID="433808685ab05362fb434a850fc9506320444c41481679ee3e002763400df5ed" exitCode=137 Sep 08 08:35:47.430901 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.430891 2585 generic.go:356] "Generic (PLEG): container finished" podID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerID="56cf5183f28f167e2c5cfb0445ba0ad4c1a8f135e786d52995859eb203137ff1" exitCode=137 Sep 08 08:35:47.431110 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.430973 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" event={"ID":"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e","Type":"ContainerDied","Data":"433808685ab05362fb434a850fc9506320444c41481679ee3e002763400df5ed"} Sep 08 08:35:47.431110 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.430985 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" Sep 08 08:35:47.431110 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.431009 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" event={"ID":"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e","Type":"ContainerDied","Data":"56cf5183f28f167e2c5cfb0445ba0ad4c1a8f135e786d52995859eb203137ff1"} Sep 08 08:35:47.431110 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.431027 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8" event={"ID":"9808c718-f0c0-45d5-ba6e-2ea76bd56d9e","Type":"ContainerDied","Data":"a747d6760067ef2458dc72a7e1e9c760e5107034f760dc1729d50cc2af8392ee"} Sep 08 08:35:47.431110 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.431044 2585 scope.go:117] "RemoveContainer" containerID="e0c5e841ecdacf75e6436e6167fae7237177bff2b0433120d96be6dd3e7058cc" Sep 08 08:35:47.439560 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.439537 2585 scope.go:117] "RemoveContainer" containerID="433808685ab05362fb434a850fc9506320444c41481679ee3e002763400df5ed" Sep 08 08:35:47.447556 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.447538 2585 scope.go:117] "RemoveContainer" containerID="2f7762d456d4dbac7514a0deecacb3851851d758df97f9ce288f7836d2555203" Sep 08 08:35:47.455111 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.455095 2585 scope.go:117] "RemoveContainer" containerID="56cf5183f28f167e2c5cfb0445ba0ad4c1a8f135e786d52995859eb203137ff1" Sep 08 08:35:47.455606 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.455587 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8"] Sep 08 08:35:47.462682 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.462663 2585 scope.go:117] "RemoveContainer" containerID="a93c1140b5518c3d4d11004b99f0c104d0beb40f2a4dd901bd9b4a0ca5e93660" Sep 08 08:35:47.469288 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.469264 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-qc9f8"] Sep 08 08:35:47.470103 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.470088 2585 scope.go:117] "RemoveContainer" containerID="666864c4c035a120891bb654a7af253eba15dd280bf152f936d95f54df0b4518" Sep 08 08:35:47.477071 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.477055 2585 scope.go:117] "RemoveContainer" containerID="e0c5e841ecdacf75e6436e6167fae7237177bff2b0433120d96be6dd3e7058cc" Sep 08 08:35:47.477335 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:35:47.477318 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0c5e841ecdacf75e6436e6167fae7237177bff2b0433120d96be6dd3e7058cc\": container with ID starting with e0c5e841ecdacf75e6436e6167fae7237177bff2b0433120d96be6dd3e7058cc not found: ID does not exist" containerID="e0c5e841ecdacf75e6436e6167fae7237177bff2b0433120d96be6dd3e7058cc" Sep 08 08:35:47.477418 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.477347 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0c5e841ecdacf75e6436e6167fae7237177bff2b0433120d96be6dd3e7058cc"} err="failed to get container status \"e0c5e841ecdacf75e6436e6167fae7237177bff2b0433120d96be6dd3e7058cc\": rpc error: code = NotFound desc = could not find container \"e0c5e841ecdacf75e6436e6167fae7237177bff2b0433120d96be6dd3e7058cc\": container with ID starting with e0c5e841ecdacf75e6436e6167fae7237177bff2b0433120d96be6dd3e7058cc not found: ID does not exist" Sep 08 08:35:47.477418 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.477402 2585 scope.go:117] "RemoveContainer" containerID="433808685ab05362fb434a850fc9506320444c41481679ee3e002763400df5ed" Sep 08 08:35:47.477683 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:35:47.477665 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"433808685ab05362fb434a850fc9506320444c41481679ee3e002763400df5ed\": container with ID starting with 433808685ab05362fb434a850fc9506320444c41481679ee3e002763400df5ed not found: ID does not exist" containerID="433808685ab05362fb434a850fc9506320444c41481679ee3e002763400df5ed" Sep 08 08:35:47.477722 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.477689 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"433808685ab05362fb434a850fc9506320444c41481679ee3e002763400df5ed"} err="failed to get container status \"433808685ab05362fb434a850fc9506320444c41481679ee3e002763400df5ed\": rpc error: code = NotFound desc = could not find container \"433808685ab05362fb434a850fc9506320444c41481679ee3e002763400df5ed\": container with ID starting with 433808685ab05362fb434a850fc9506320444c41481679ee3e002763400df5ed not found: ID does not exist" Sep 08 08:35:47.477722 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.477705 2585 scope.go:117] "RemoveContainer" containerID="2f7762d456d4dbac7514a0deecacb3851851d758df97f9ce288f7836d2555203" Sep 08 08:35:47.477924 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:35:47.477907 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f7762d456d4dbac7514a0deecacb3851851d758df97f9ce288f7836d2555203\": container with ID starting with 2f7762d456d4dbac7514a0deecacb3851851d758df97f9ce288f7836d2555203 not found: ID does not exist" containerID="2f7762d456d4dbac7514a0deecacb3851851d758df97f9ce288f7836d2555203" Sep 08 08:35:47.477988 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.477932 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f7762d456d4dbac7514a0deecacb3851851d758df97f9ce288f7836d2555203"} err="failed to get container status \"2f7762d456d4dbac7514a0deecacb3851851d758df97f9ce288f7836d2555203\": rpc error: code = NotFound desc = could not find container \"2f7762d456d4dbac7514a0deecacb3851851d758df97f9ce288f7836d2555203\": container with ID starting with 2f7762d456d4dbac7514a0deecacb3851851d758df97f9ce288f7836d2555203 not found: ID does not exist" Sep 08 08:35:47.477988 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.477951 2585 scope.go:117] "RemoveContainer" containerID="56cf5183f28f167e2c5cfb0445ba0ad4c1a8f135e786d52995859eb203137ff1" Sep 08 08:35:47.478179 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:35:47.478163 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56cf5183f28f167e2c5cfb0445ba0ad4c1a8f135e786d52995859eb203137ff1\": container with ID starting with 56cf5183f28f167e2c5cfb0445ba0ad4c1a8f135e786d52995859eb203137ff1 not found: ID does not exist" containerID="56cf5183f28f167e2c5cfb0445ba0ad4c1a8f135e786d52995859eb203137ff1" Sep 08 08:35:47.478213 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.478185 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56cf5183f28f167e2c5cfb0445ba0ad4c1a8f135e786d52995859eb203137ff1"} err="failed to get container status \"56cf5183f28f167e2c5cfb0445ba0ad4c1a8f135e786d52995859eb203137ff1\": rpc error: code = NotFound desc = could not find container \"56cf5183f28f167e2c5cfb0445ba0ad4c1a8f135e786d52995859eb203137ff1\": container with ID starting with 56cf5183f28f167e2c5cfb0445ba0ad4c1a8f135e786d52995859eb203137ff1 not found: ID does not exist" Sep 08 08:35:47.478213 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.478204 2585 scope.go:117] "RemoveContainer" containerID="a93c1140b5518c3d4d11004b99f0c104d0beb40f2a4dd901bd9b4a0ca5e93660" Sep 08 08:35:47.478432 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:35:47.478416 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a93c1140b5518c3d4d11004b99f0c104d0beb40f2a4dd901bd9b4a0ca5e93660\": container with ID starting with a93c1140b5518c3d4d11004b99f0c104d0beb40f2a4dd901bd9b4a0ca5e93660 not found: ID does not exist" containerID="a93c1140b5518c3d4d11004b99f0c104d0beb40f2a4dd901bd9b4a0ca5e93660" Sep 08 08:35:47.478506 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.478438 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a93c1140b5518c3d4d11004b99f0c104d0beb40f2a4dd901bd9b4a0ca5e93660"} err="failed to get container status \"a93c1140b5518c3d4d11004b99f0c104d0beb40f2a4dd901bd9b4a0ca5e93660\": rpc error: code = NotFound desc = could not find container \"a93c1140b5518c3d4d11004b99f0c104d0beb40f2a4dd901bd9b4a0ca5e93660\": container with ID starting with a93c1140b5518c3d4d11004b99f0c104d0beb40f2a4dd901bd9b4a0ca5e93660 not found: ID does not exist" Sep 08 08:35:47.478506 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.478458 2585 scope.go:117] "RemoveContainer" containerID="666864c4c035a120891bb654a7af253eba15dd280bf152f936d95f54df0b4518" Sep 08 08:35:47.478686 ip-10-0-143-1 kubenswrapper[2585]: E0908 08:35:47.478670 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"666864c4c035a120891bb654a7af253eba15dd280bf152f936d95f54df0b4518\": container with ID starting with 666864c4c035a120891bb654a7af253eba15dd280bf152f936d95f54df0b4518 not found: ID does not exist" containerID="666864c4c035a120891bb654a7af253eba15dd280bf152f936d95f54df0b4518" Sep 08 08:35:47.478746 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.478692 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"666864c4c035a120891bb654a7af253eba15dd280bf152f936d95f54df0b4518"} err="failed to get container status \"666864c4c035a120891bb654a7af253eba15dd280bf152f936d95f54df0b4518\": rpc error: code = NotFound desc = could not find container \"666864c4c035a120891bb654a7af253eba15dd280bf152f936d95f54df0b4518\": container with ID starting with 666864c4c035a120891bb654a7af253eba15dd280bf152f936d95f54df0b4518 not found: ID does not exist" Sep 08 08:35:47.478746 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.478710 2585 scope.go:117] "RemoveContainer" containerID="e0c5e841ecdacf75e6436e6167fae7237177bff2b0433120d96be6dd3e7058cc" Sep 08 08:35:47.478938 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.478909 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0c5e841ecdacf75e6436e6167fae7237177bff2b0433120d96be6dd3e7058cc"} err="failed to get container status \"e0c5e841ecdacf75e6436e6167fae7237177bff2b0433120d96be6dd3e7058cc\": rpc error: code = NotFound desc = could not find container \"e0c5e841ecdacf75e6436e6167fae7237177bff2b0433120d96be6dd3e7058cc\": container with ID starting with e0c5e841ecdacf75e6436e6167fae7237177bff2b0433120d96be6dd3e7058cc not found: ID does not exist" Sep 08 08:35:47.479006 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.478940 2585 scope.go:117] "RemoveContainer" containerID="433808685ab05362fb434a850fc9506320444c41481679ee3e002763400df5ed" Sep 08 08:35:47.479141 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.479124 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"433808685ab05362fb434a850fc9506320444c41481679ee3e002763400df5ed"} err="failed to get container status \"433808685ab05362fb434a850fc9506320444c41481679ee3e002763400df5ed\": rpc error: code = NotFound desc = could not find container \"433808685ab05362fb434a850fc9506320444c41481679ee3e002763400df5ed\": container with ID starting with 433808685ab05362fb434a850fc9506320444c41481679ee3e002763400df5ed not found: ID does not exist" Sep 08 08:35:47.479206 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.479143 2585 scope.go:117] "RemoveContainer" containerID="2f7762d456d4dbac7514a0deecacb3851851d758df97f9ce288f7836d2555203" Sep 08 08:35:47.479342 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.479326 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f7762d456d4dbac7514a0deecacb3851851d758df97f9ce288f7836d2555203"} err="failed to get container status \"2f7762d456d4dbac7514a0deecacb3851851d758df97f9ce288f7836d2555203\": rpc error: code = NotFound desc = could not find container \"2f7762d456d4dbac7514a0deecacb3851851d758df97f9ce288f7836d2555203\": container with ID starting with 2f7762d456d4dbac7514a0deecacb3851851d758df97f9ce288f7836d2555203 not found: ID does not exist" Sep 08 08:35:47.479407 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.479342 2585 scope.go:117] "RemoveContainer" containerID="56cf5183f28f167e2c5cfb0445ba0ad4c1a8f135e786d52995859eb203137ff1" Sep 08 08:35:47.479679 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.479663 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56cf5183f28f167e2c5cfb0445ba0ad4c1a8f135e786d52995859eb203137ff1"} err="failed to get container status \"56cf5183f28f167e2c5cfb0445ba0ad4c1a8f135e786d52995859eb203137ff1\": rpc error: code = NotFound desc = could not find container \"56cf5183f28f167e2c5cfb0445ba0ad4c1a8f135e786d52995859eb203137ff1\": container with ID starting with 56cf5183f28f167e2c5cfb0445ba0ad4c1a8f135e786d52995859eb203137ff1 not found: ID does not exist" Sep 08 08:35:47.479743 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.479680 2585 scope.go:117] "RemoveContainer" containerID="a93c1140b5518c3d4d11004b99f0c104d0beb40f2a4dd901bd9b4a0ca5e93660" Sep 08 08:35:47.479909 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.479891 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a93c1140b5518c3d4d11004b99f0c104d0beb40f2a4dd901bd9b4a0ca5e93660"} err="failed to get container status \"a93c1140b5518c3d4d11004b99f0c104d0beb40f2a4dd901bd9b4a0ca5e93660\": rpc error: code = NotFound desc = could not find container \"a93c1140b5518c3d4d11004b99f0c104d0beb40f2a4dd901bd9b4a0ca5e93660\": container with ID starting with a93c1140b5518c3d4d11004b99f0c104d0beb40f2a4dd901bd9b4a0ca5e93660 not found: ID does not exist" Sep 08 08:35:47.479956 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.479910 2585 scope.go:117] "RemoveContainer" containerID="666864c4c035a120891bb654a7af253eba15dd280bf152f936d95f54df0b4518" Sep 08 08:35:47.480142 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:47.480124 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"666864c4c035a120891bb654a7af253eba15dd280bf152f936d95f54df0b4518"} err="failed to get container status \"666864c4c035a120891bb654a7af253eba15dd280bf152f936d95f54df0b4518\": rpc error: code = NotFound desc = could not find container \"666864c4c035a120891bb654a7af253eba15dd280bf152f936d95f54df0b4518\": container with ID starting with 666864c4c035a120891bb654a7af253eba15dd280bf152f936d95f54df0b4518 not found: ID does not exist" Sep 08 08:35:49.175190 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:35:49.175147 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" path="/var/lib/kubelet/pods/9808c718-f0c0-45d5-ba6e-2ea76bd56d9e/volumes" Sep 08 08:36:41.785178 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.785101 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf"] Sep 08 08:36:41.785646 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.785411 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerName="offline" Sep 08 08:36:41.785646 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.785422 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerName="offline" Sep 08 08:36:41.785646 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.785429 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerName="ui" Sep 08 08:36:41.785646 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.785434 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerName="ui" Sep 08 08:36:41.785646 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.785442 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerName="registry" Sep 08 08:36:41.785646 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.785447 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerName="registry" Sep 08 08:36:41.785646 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.785453 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerName="online" Sep 08 08:36:41.785646 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.785458 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerName="online" Sep 08 08:36:41.785646 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.785478 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerName="feast-init" Sep 08 08:36:41.785646 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.785483 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerName="feast-init" Sep 08 08:36:41.785646 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.785489 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerName="feast-apply" Sep 08 08:36:41.785646 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.785494 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerName="feast-apply" Sep 08 08:36:41.785646 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.785536 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerName="online" Sep 08 08:36:41.785646 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.785547 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerName="offline" Sep 08 08:36:41.785646 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.785553 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerName="registry" Sep 08 08:36:41.785646 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.785560 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="9808c718-f0c0-45d5-ba6e-2ea76bd56d9e" containerName="ui" Sep 08 08:36:41.788863 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.788846 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" Sep 08 08:36:41.791503 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.791480 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-ppddh\"" Sep 08 08:36:41.791628 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.791592 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 08 08:36:41.791628 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.791620 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 08 08:36:41.791701 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.791592 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 08 08:36:41.792679 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.792662 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 08 08:36:41.792791 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.792718 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 08 08:36:41.792854 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.792814 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 08 08:36:41.800821 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.800801 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf"] Sep 08 08:36:41.856211 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.856175 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9gmx\" (UniqueName: \"kubernetes.io/projected/dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3-kube-api-access-z9gmx\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-dcwnf\" (UID: \"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" Sep 08 08:36:41.856373 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.856273 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-dcwnf\" (UID: \"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" Sep 08 08:36:41.856373 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.856316 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-dcwnf\" (UID: \"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" Sep 08 08:36:41.856373 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.856345 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-dcwnf\" (UID: \"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" Sep 08 08:36:41.856373 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.856365 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-dcwnf\" (UID: \"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" Sep 08 08:36:41.856549 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.856429 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-dcwnf\" (UID: \"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" Sep 08 08:36:41.957285 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.957254 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9gmx\" (UniqueName: \"kubernetes.io/projected/dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3-kube-api-access-z9gmx\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-dcwnf\" (UID: \"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" Sep 08 08:36:41.957511 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.957302 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-dcwnf\" (UID: \"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" Sep 08 08:36:41.957511 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.957424 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-dcwnf\" (UID: \"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" Sep 08 08:36:41.957511 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.957460 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-dcwnf\" (UID: \"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" Sep 08 08:36:41.957511 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.957498 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-dcwnf\" (UID: \"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" Sep 08 08:36:41.957720 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.957525 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-dcwnf\" (UID: \"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" Sep 08 08:36:41.957720 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.957632 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-dcwnf\" (UID: \"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" Sep 08 08:36:41.958137 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.958113 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-dcwnf\" (UID: \"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" Sep 08 08:36:41.959969 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.959939 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-dcwnf\" (UID: \"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" Sep 08 08:36:41.960056 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.960009 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-dcwnf\" (UID: \"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" Sep 08 08:36:41.960056 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.960040 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-dcwnf\" (UID: \"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" Sep 08 08:36:41.965702 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:41.965683 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9gmx\" (UniqueName: \"kubernetes.io/projected/dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3-kube-api-access-z9gmx\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-dcwnf\" (UID: \"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" Sep 08 08:36:42.098627 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:42.098545 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" Sep 08 08:36:42.382096 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:42.382069 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf"] Sep 08 08:36:42.384828 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:36:42.384806 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddc9b3ea0_f3c5_41a7_8092_d9d49dc83be3.slice/crio-39604aa02b6796b60c8d9f6d6d69e959e5c0660f72bd9d6930fdc9d3f03d60ce WatchSource:0}: Error finding container 39604aa02b6796b60c8d9f6d6d69e959e5c0660f72bd9d6930fdc9d3f03d60ce: Status 404 returned error can't find the container with id 39604aa02b6796b60c8d9f6d6d69e959e5c0660f72bd9d6930fdc9d3f03d60ce Sep 08 08:36:42.645250 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:42.645156 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" event={"ID":"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3","Type":"ContainerStarted","Data":"e38a13d995fbdc522eb32049e8069af60639001e7ed1610877b02a77453159de"} Sep 08 08:36:42.645250 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:42.645220 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" event={"ID":"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3","Type":"ContainerStarted","Data":"39604aa02b6796b60c8d9f6d6d69e959e5c0660f72bd9d6930fdc9d3f03d60ce"} Sep 08 08:36:47.667465 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:47.667430 2585 generic.go:356] "Generic (PLEG): container finished" podID="dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" containerID="e38a13d995fbdc522eb32049e8069af60639001e7ed1610877b02a77453159de" exitCode=0 Sep 08 08:36:47.667913 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:47.667512 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" event={"ID":"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3","Type":"ContainerDied","Data":"e38a13d995fbdc522eb32049e8069af60639001e7ed1610877b02a77453159de"} Sep 08 08:36:48.673623 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:36:48.673589 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" event={"ID":"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3","Type":"ContainerStarted","Data":"3488a356a9ef0ab6b0aea4e5e27177013a4e03376dc86e61f01845a985f54aa2"} Sep 08 08:37:11.768312 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:37:11.768270 2585 generic.go:356] "Generic (PLEG): container finished" podID="dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" containerID="3488a356a9ef0ab6b0aea4e5e27177013a4e03376dc86e61f01845a985f54aa2" exitCode=0 Sep 08 08:37:11.768749 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:37:11.768337 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" event={"ID":"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3","Type":"ContainerDied","Data":"3488a356a9ef0ab6b0aea4e5e27177013a4e03376dc86e61f01845a985f54aa2"} Sep 08 08:37:12.776607 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:37:12.776562 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" event={"ID":"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3","Type":"ContainerStarted","Data":"a8e7eaf7ab67539db79a4b9b7ebf1337f68873f190954f2bb8dee1fd7ad8fc59"} Sep 08 08:37:12.777103 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:37:12.776627 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" event={"ID":"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3","Type":"ContainerStarted","Data":"d39c224b46b99e267334c90f52b75363159814513f7ecf8660795473627299ae"} Sep 08 08:37:12.777103 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:37:12.776648 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" event={"ID":"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3","Type":"ContainerStarted","Data":"5559cd3106a0450423f647610ee34db0f49fe170ec7efda1d1651d7a0e1f821c"} Sep 08 08:37:12.804080 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:37:12.804011 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" podStartSLOduration=31.803988201 podStartE2EDuration="31.803988201s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 08:36:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:37:12.80249641 +0000 UTC m=+1646.140136051" watchObservedRunningTime="2026-09-08 08:37:12.803988201 +0000 UTC m=+1646.141627842" Sep 08 08:37:15.099343 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:37:15.099304 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" Sep 08 08:37:15.099922 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:37:15.099395 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" Sep 08 08:37:15.099922 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:37:15.099420 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" Sep 08 08:37:15.101430 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:37:15.101395 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" podUID="dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.22:8816: connect: connection refused" Sep 08 08:37:15.101564 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:37:15.101438 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" podUID="dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" containerName="online" probeResult="failure" output="Get \"https://10.134.0.22:6567/health\": dial tcp 10.134.0.22:6567: connect: connection refused" Sep 08 08:37:15.102115 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:37:15.102091 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" podUID="dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.22:8443: connect: connection refused" Sep 08 08:37:18.099364 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:37:18.099313 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" podUID="dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.22:8816: connect: connection refused" Sep 08 08:37:18.099850 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:37:18.099370 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" podUID="dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" containerName="online" probeResult="failure" output="Get \"https://10.134.0.22:6567/health\": dial tcp 10.134.0.22:6567: connect: connection refused" Sep 08 08:37:18.099850 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:37:18.099321 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" podUID="dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.22:8443: connect: connection refused" Sep 08 08:37:21.100672 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:37:21.100635 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" Sep 08 08:37:21.101048 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:37:21.100772 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" Sep 08 08:37:21.101202 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:37:21.101183 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" Sep 08 08:37:21.101295 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:37:21.101218 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" Sep 08 08:37:21.101457 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:37:21.101441 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" Sep 08 08:37:21.101521 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:37:21.101487 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" Sep 08 08:37:22.098986 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:37:22.098949 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" Sep 08 08:37:22.104506 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:37:22.104468 2585 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" podUID="dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" containerName="online" probeResult="failure" output="Get \"https://10.134.0.22:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 08 08:37:24.845097 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:37:24.845067 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" Sep 08 08:37:25.585799 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:37:25.585767 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" Sep 08 08:38:23.103568 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:23.103480 2585 prober.go:120] "Probe failed" probeType="Liveness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" podUID="dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" containerName="online" probeResult="failure" output="Get \"https://10.134.0.22:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 08 08:38:27.772539 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:27.772505 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf"] Sep 08 08:38:27.772953 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:27.772755 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" podUID="dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" containerName="online" containerID="cri-o://5559cd3106a0450423f647610ee34db0f49fe170ec7efda1d1651d7a0e1f821c" gracePeriod=30 Sep 08 08:38:27.772953 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:27.772784 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" podUID="dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" containerName="ui" containerID="cri-o://a8e7eaf7ab67539db79a4b9b7ebf1337f68873f190954f2bb8dee1fd7ad8fc59" gracePeriod=30 Sep 08 08:38:27.779062 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:27.773518 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" podUID="dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" containerName="offline" containerID="cri-o://d39c224b46b99e267334c90f52b75363159814513f7ecf8660795473627299ae" gracePeriod=30 Sep 08 08:38:27.999397 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:27.999344 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m"] Sep 08 08:38:27.999778 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:27.999731 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" podUID="f6bf283b-17f0-49a1-9782-96af409728d1" containerName="ui" containerID="cri-o://e75bc372fe53bdb19d4d8b76e85bdb530e4c09c00bb78f27d8ede50a23daca8c" gracePeriod=30 Sep 08 08:38:27.999778 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:27.999673 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" podUID="f6bf283b-17f0-49a1-9782-96af409728d1" containerName="registry" containerID="cri-o://0db7f8cecb90169d39a71cb14fb0708a4d20ac3153583c8bc2573ba78eb23978" gracePeriod=30 Sep 08 08:38:27.999778 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:27.999741 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" podUID="f6bf283b-17f0-49a1-9782-96af409728d1" containerName="offline" containerID="cri-o://f15fe4dc98a7ba988a917adb6c4858ad02431aacc387c22031d80275abc08c99" gracePeriod=30 Sep 08 08:38:27.999778 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:27.999771 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" podUID="f6bf283b-17f0-49a1-9782-96af409728d1" containerName="online" containerID="cri-o://c725a2ad4e2803369f399ba7c3f6338fe9db60a09978c49f21b76f0a66dfc9cd" gracePeriod=30 Sep 08 08:38:28.888828 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:28.888789 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" podUID="f6bf283b-17f0-49a1-9782-96af409728d1" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.21:8443: connect: connection refused" Sep 08 08:38:29.107792 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:29.107758 2585 generic.go:356] "Generic (PLEG): container finished" podID="dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" containerID="a8e7eaf7ab67539db79a4b9b7ebf1337f68873f190954f2bb8dee1fd7ad8fc59" exitCode=0 Sep 08 08:38:29.107792 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:29.107781 2585 generic.go:356] "Generic (PLEG): container finished" podID="dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" containerID="5559cd3106a0450423f647610ee34db0f49fe170ec7efda1d1651d7a0e1f821c" exitCode=0 Sep 08 08:38:29.107985 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:29.107828 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" event={"ID":"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3","Type":"ContainerDied","Data":"a8e7eaf7ab67539db79a4b9b7ebf1337f68873f190954f2bb8dee1fd7ad8fc59"} Sep 08 08:38:29.107985 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:29.107871 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" event={"ID":"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3","Type":"ContainerDied","Data":"5559cd3106a0450423f647610ee34db0f49fe170ec7efda1d1651d7a0e1f821c"} Sep 08 08:38:29.110719 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:29.110692 2585 generic.go:356] "Generic (PLEG): container finished" podID="f6bf283b-17f0-49a1-9782-96af409728d1" containerID="e75bc372fe53bdb19d4d8b76e85bdb530e4c09c00bb78f27d8ede50a23daca8c" exitCode=0 Sep 08 08:38:29.110719 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:29.110713 2585 generic.go:356] "Generic (PLEG): container finished" podID="f6bf283b-17f0-49a1-9782-96af409728d1" containerID="c725a2ad4e2803369f399ba7c3f6338fe9db60a09978c49f21b76f0a66dfc9cd" exitCode=0 Sep 08 08:38:29.110878 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:29.110749 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" event={"ID":"f6bf283b-17f0-49a1-9782-96af409728d1","Type":"ContainerDied","Data":"e75bc372fe53bdb19d4d8b76e85bdb530e4c09c00bb78f27d8ede50a23daca8c"} Sep 08 08:38:29.110878 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:29.110774 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" event={"ID":"f6bf283b-17f0-49a1-9782-96af409728d1","Type":"ContainerDied","Data":"c725a2ad4e2803369f399ba7c3f6338fe9db60a09978c49f21b76f0a66dfc9cd"} Sep 08 08:38:29.397309 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:29.397276 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" podUID="f6bf283b-17f0-49a1-9782-96af409728d1" containerName="online" probeResult="failure" output="Get \"https://10.134.0.21:6567/health\": dial tcp 10.134.0.21:6567: connect: connection refused" Sep 08 08:38:31.101658 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:31.101616 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" podUID="dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.22:8443: connect: connection refused" Sep 08 08:38:34.846119 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:34.846077 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" podUID="dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" containerName="online" probeResult="failure" output="Get \"https://10.134.0.22:6567/health\": dial tcp 10.134.0.22:6567: connect: connection refused" Sep 08 08:38:38.889816 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:38.889766 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" podUID="f6bf283b-17f0-49a1-9782-96af409728d1" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.21:8443: connect: connection refused" Sep 08 08:38:39.397761 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:39.397720 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" podUID="f6bf283b-17f0-49a1-9782-96af409728d1" containerName="online" probeResult="failure" output="Get \"https://10.134.0.21:6567/health\": dial tcp 10.134.0.21:6567: connect: connection refused" Sep 08 08:38:41.102115 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:41.102076 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" podUID="dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.22:8443: connect: connection refused" Sep 08 08:38:44.845730 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:44.845678 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" podUID="dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" containerName="online" probeResult="failure" output="Get \"https://10.134.0.22:6567/health\": dial tcp 10.134.0.22:6567: connect: connection refused" Sep 08 08:38:48.889529 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:48.889475 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" podUID="f6bf283b-17f0-49a1-9782-96af409728d1" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.21:8443: connect: connection refused" Sep 08 08:38:48.889966 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:48.889642 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:38:49.397311 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:49.397270 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" podUID="f6bf283b-17f0-49a1-9782-96af409728d1" containerName="online" probeResult="failure" output="Get \"https://10.134.0.21:6567/health\": dial tcp 10.134.0.21:6567: connect: connection refused" Sep 08 08:38:49.397514 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:49.397465 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:38:51.101328 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:51.101286 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" podUID="dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.22:8443: connect: connection refused" Sep 08 08:38:51.101779 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:51.101476 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" Sep 08 08:38:54.846065 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:54.846015 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" podUID="dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" containerName="online" probeResult="failure" output="Get \"https://10.134.0.22:6567/health\": dial tcp 10.134.0.22:6567: connect: connection refused" Sep 08 08:38:54.846642 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:54.846190 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" Sep 08 08:38:58.231769 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.231729 2585 generic.go:356] "Generic (PLEG): container finished" podID="f6bf283b-17f0-49a1-9782-96af409728d1" containerID="f15fe4dc98a7ba988a917adb6c4858ad02431aacc387c22031d80275abc08c99" exitCode=137 Sep 08 08:38:58.231769 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.231756 2585 generic.go:356] "Generic (PLEG): container finished" podID="f6bf283b-17f0-49a1-9782-96af409728d1" containerID="0db7f8cecb90169d39a71cb14fb0708a4d20ac3153583c8bc2573ba78eb23978" exitCode=137 Sep 08 08:38:58.232220 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.231792 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" event={"ID":"f6bf283b-17f0-49a1-9782-96af409728d1","Type":"ContainerDied","Data":"f15fe4dc98a7ba988a917adb6c4858ad02431aacc387c22031d80275abc08c99"} Sep 08 08:38:58.232220 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.231838 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" event={"ID":"f6bf283b-17f0-49a1-9782-96af409728d1","Type":"ContainerDied","Data":"0db7f8cecb90169d39a71cb14fb0708a4d20ac3153583c8bc2573ba78eb23978"} Sep 08 08:38:58.234597 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.234573 2585 generic.go:356] "Generic (PLEG): container finished" podID="dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" containerID="d39c224b46b99e267334c90f52b75363159814513f7ecf8660795473627299ae" exitCode=137 Sep 08 08:38:58.234753 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.234639 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" event={"ID":"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3","Type":"ContainerDied","Data":"d39c224b46b99e267334c90f52b75363159814513f7ecf8660795473627299ae"} Sep 08 08:38:58.424526 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.424502 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" Sep 08 08:38:58.538622 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.538590 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9gmx\" (UniqueName: \"kubernetes.io/projected/dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3-kube-api-access-z9gmx\") pod \"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3\" (UID: \"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3\") " Sep 08 08:38:58.538763 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.538655 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3-online-tls\") pod \"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3\" (UID: \"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3\") " Sep 08 08:38:58.538763 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.538700 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3-feast-data\") pod \"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3\" (UID: \"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3\") " Sep 08 08:38:58.538763 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.538724 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3-registry-tls\") pod \"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3\" (UID: \"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3\") " Sep 08 08:38:58.538763 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.538752 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3-offline-tls\") pod \"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3\" (UID: \"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3\") " Sep 08 08:38:58.538986 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.538799 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3-ui-tls\") pod \"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3\" (UID: \"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3\") " Sep 08 08:38:58.539156 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.539122 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" (UID: "dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 08:38:58.539279 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.539222 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3-feast-data" (OuterVolumeSpecName: "feast-data") pod "dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" (UID: "dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 08:38:58.541156 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.541126 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" (UID: "dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:38:58.541243 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.541194 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3-kube-api-access-z9gmx" (OuterVolumeSpecName: "kube-api-access-z9gmx") pod "dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" (UID: "dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3"). InnerVolumeSpecName "kube-api-access-z9gmx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 08:38:58.541243 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.541236 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" (UID: "dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:38:58.541325 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.541309 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3-online-tls" (OuterVolumeSpecName: "online-tls") pod "dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" (UID: "dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:38:58.639620 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.639596 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9gmx\" (UniqueName: \"kubernetes.io/projected/dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3-kube-api-access-z9gmx\") on node \"ip-10-0-143-1.ec2.internal\" DevicePath \"\"" Sep 08 08:38:58.639620 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.639620 2585 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3-online-tls\") on node \"ip-10-0-143-1.ec2.internal\" DevicePath \"\"" Sep 08 08:38:58.639792 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.639629 2585 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3-feast-data\") on node \"ip-10-0-143-1.ec2.internal\" DevicePath \"\"" Sep 08 08:38:58.639792 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.639637 2585 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3-registry-tls\") on node \"ip-10-0-143-1.ec2.internal\" DevicePath \"\"" Sep 08 08:38:58.639792 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.639644 2585 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3-offline-tls\") on node \"ip-10-0-143-1.ec2.internal\" DevicePath \"\"" Sep 08 08:38:58.639792 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.639651 2585 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3-ui-tls\") on node \"ip-10-0-143-1.ec2.internal\" DevicePath \"\"" Sep 08 08:38:58.641600 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.641584 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:38:58.740625 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.740587 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f6bf283b-17f0-49a1-9782-96af409728d1-online-tls\") pod \"f6bf283b-17f0-49a1-9782-96af409728d1\" (UID: \"f6bf283b-17f0-49a1-9782-96af409728d1\") " Sep 08 08:38:58.740625 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.740623 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f6bf283b-17f0-49a1-9782-96af409728d1-registry-tls\") pod \"f6bf283b-17f0-49a1-9782-96af409728d1\" (UID: \"f6bf283b-17f0-49a1-9782-96af409728d1\") " Sep 08 08:38:58.740882 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.740642 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f6bf283b-17f0-49a1-9782-96af409728d1-ui-tls\") pod \"f6bf283b-17f0-49a1-9782-96af409728d1\" (UID: \"f6bf283b-17f0-49a1-9782-96af409728d1\") " Sep 08 08:38:58.740882 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.740671 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82r7z\" (UniqueName: \"kubernetes.io/projected/f6bf283b-17f0-49a1-9782-96af409728d1-kube-api-access-82r7z\") pod \"f6bf283b-17f0-49a1-9782-96af409728d1\" (UID: \"f6bf283b-17f0-49a1-9782-96af409728d1\") " Sep 08 08:38:58.740882 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.740688 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f6bf283b-17f0-49a1-9782-96af409728d1-feast-data\") pod \"f6bf283b-17f0-49a1-9782-96af409728d1\" (UID: \"f6bf283b-17f0-49a1-9782-96af409728d1\") " Sep 08 08:38:58.740882 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.740731 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f6bf283b-17f0-49a1-9782-96af409728d1-offline-tls\") pod \"f6bf283b-17f0-49a1-9782-96af409728d1\" (UID: \"f6bf283b-17f0-49a1-9782-96af409728d1\") " Sep 08 08:38:58.741370 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.741318 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6bf283b-17f0-49a1-9782-96af409728d1-feast-data" (OuterVolumeSpecName: "feast-data") pod "f6bf283b-17f0-49a1-9782-96af409728d1" (UID: "f6bf283b-17f0-49a1-9782-96af409728d1"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 08:38:58.743081 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.743049 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6bf283b-17f0-49a1-9782-96af409728d1-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "f6bf283b-17f0-49a1-9782-96af409728d1" (UID: "f6bf283b-17f0-49a1-9782-96af409728d1"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:38:58.743178 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.743078 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6bf283b-17f0-49a1-9782-96af409728d1-online-tls" (OuterVolumeSpecName: "online-tls") pod "f6bf283b-17f0-49a1-9782-96af409728d1" (UID: "f6bf283b-17f0-49a1-9782-96af409728d1"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:38:58.743178 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.743133 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6bf283b-17f0-49a1-9782-96af409728d1-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f6bf283b-17f0-49a1-9782-96af409728d1" (UID: "f6bf283b-17f0-49a1-9782-96af409728d1"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:38:58.743440 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.743421 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6bf283b-17f0-49a1-9782-96af409728d1-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "f6bf283b-17f0-49a1-9782-96af409728d1" (UID: "f6bf283b-17f0-49a1-9782-96af409728d1"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:38:58.743529 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.743440 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6bf283b-17f0-49a1-9782-96af409728d1-kube-api-access-82r7z" (OuterVolumeSpecName: "kube-api-access-82r7z") pod "f6bf283b-17f0-49a1-9782-96af409728d1" (UID: "f6bf283b-17f0-49a1-9782-96af409728d1"). InnerVolumeSpecName "kube-api-access-82r7z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 08:38:58.842133 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.842059 2585 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f6bf283b-17f0-49a1-9782-96af409728d1-online-tls\") on node \"ip-10-0-143-1.ec2.internal\" DevicePath \"\"" Sep 08 08:38:58.842133 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.842085 2585 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f6bf283b-17f0-49a1-9782-96af409728d1-registry-tls\") on node \"ip-10-0-143-1.ec2.internal\" DevicePath \"\"" Sep 08 08:38:58.842133 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.842092 2585 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f6bf283b-17f0-49a1-9782-96af409728d1-ui-tls\") on node \"ip-10-0-143-1.ec2.internal\" DevicePath \"\"" Sep 08 08:38:58.842133 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.842102 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82r7z\" (UniqueName: \"kubernetes.io/projected/f6bf283b-17f0-49a1-9782-96af409728d1-kube-api-access-82r7z\") on node \"ip-10-0-143-1.ec2.internal\" DevicePath \"\"" Sep 08 08:38:58.842133 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.842113 2585 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f6bf283b-17f0-49a1-9782-96af409728d1-feast-data\") on node \"ip-10-0-143-1.ec2.internal\" DevicePath \"\"" Sep 08 08:38:58.842133 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:58.842122 2585 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f6bf283b-17f0-49a1-9782-96af409728d1-offline-tls\") on node \"ip-10-0-143-1.ec2.internal\" DevicePath \"\"" Sep 08 08:38:59.241067 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:59.241030 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" Sep 08 08:38:59.241524 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:59.241027 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m" event={"ID":"f6bf283b-17f0-49a1-9782-96af409728d1","Type":"ContainerDied","Data":"713f3779472a9aa5e98894920760a9848305a09d615ad9e15c762bc0242f4d2a"} Sep 08 08:38:59.241524 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:59.241167 2585 scope.go:117] "RemoveContainer" containerID="e75bc372fe53bdb19d4d8b76e85bdb530e4c09c00bb78f27d8ede50a23daca8c" Sep 08 08:38:59.244029 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:59.244005 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" event={"ID":"dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3","Type":"ContainerDied","Data":"39604aa02b6796b60c8d9f6d6d69e959e5c0660f72bd9d6930fdc9d3f03d60ce"} Sep 08 08:38:59.244296 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:59.244274 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf" Sep 08 08:38:59.250747 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:59.250721 2585 scope.go:117] "RemoveContainer" containerID="f15fe4dc98a7ba988a917adb6c4858ad02431aacc387c22031d80275abc08c99" Sep 08 08:38:59.258655 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:59.258620 2585 scope.go:117] "RemoveContainer" containerID="c725a2ad4e2803369f399ba7c3f6338fe9db60a09978c49f21b76f0a66dfc9cd" Sep 08 08:38:59.266117 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:59.266102 2585 scope.go:117] "RemoveContainer" containerID="0db7f8cecb90169d39a71cb14fb0708a4d20ac3153583c8bc2573ba78eb23978" Sep 08 08:38:59.273195 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:59.273171 2585 scope.go:117] "RemoveContainer" containerID="8b8bee39da240da92127eeff280f9f0725af6c62513ee00fa4254315e936aa76" Sep 08 08:38:59.280485 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:59.280325 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m"] Sep 08 08:38:59.280550 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:59.280489 2585 scope.go:117] "RemoveContainer" containerID="2b05ae10a934fdd70b2200be8d026f780ca596edadb72eac94174577b270a1a3" Sep 08 08:38:59.288007 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:59.287988 2585 scope.go:117] "RemoveContainer" containerID="a8e7eaf7ab67539db79a4b9b7ebf1337f68873f190954f2bb8dee1fd7ad8fc59" Sep 08 08:38:59.288219 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:59.288195 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-nfx5m"] Sep 08 08:38:59.295180 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:59.295162 2585 scope.go:117] "RemoveContainer" containerID="d39c224b46b99e267334c90f52b75363159814513f7ecf8660795473627299ae" Sep 08 08:38:59.302454 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:59.302439 2585 scope.go:117] "RemoveContainer" containerID="5559cd3106a0450423f647610ee34db0f49fe170ec7efda1d1651d7a0e1f821c" Sep 08 08:38:59.309475 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:59.309454 2585 scope.go:117] "RemoveContainer" containerID="3488a356a9ef0ab6b0aea4e5e27177013a4e03376dc86e61f01845a985f54aa2" Sep 08 08:38:59.316636 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:59.316618 2585 scope.go:117] "RemoveContainer" containerID="e38a13d995fbdc522eb32049e8069af60639001e7ed1610877b02a77453159de" Sep 08 08:38:59.318289 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:59.318268 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf"] Sep 08 08:38:59.326884 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:38:59.326864 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-dcwnf"] Sep 08 08:39:01.174292 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:01.174248 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" path="/var/lib/kubelet/pods/dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3/volumes" Sep 08 08:39:01.174996 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:01.174973 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6bf283b-17f0-49a1-9782-96af409728d1" path="/var/lib/kubelet/pods/f6bf283b-17f0-49a1-9782-96af409728d1/volumes" Sep 08 08:39:36.932484 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:36.932445 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nqvlb/must-gather-qncfw"] Sep 08 08:39:36.932929 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:36.932739 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" containerName="feast-init" Sep 08 08:39:36.932929 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:36.932749 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" containerName="feast-init" Sep 08 08:39:36.932929 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:36.932757 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" containerName="offline" Sep 08 08:39:36.932929 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:36.932763 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" containerName="offline" Sep 08 08:39:36.932929 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:36.932771 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" containerName="ui" Sep 08 08:39:36.932929 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:36.932777 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" containerName="ui" Sep 08 08:39:36.932929 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:36.932785 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6bf283b-17f0-49a1-9782-96af409728d1" containerName="online" Sep 08 08:39:36.932929 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:36.932790 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6bf283b-17f0-49a1-9782-96af409728d1" containerName="online" Sep 08 08:39:36.932929 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:36.932797 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6bf283b-17f0-49a1-9782-96af409728d1" containerName="feast-init" Sep 08 08:39:36.932929 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:36.932802 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6bf283b-17f0-49a1-9782-96af409728d1" containerName="feast-init" Sep 08 08:39:36.932929 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:36.932810 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" containerName="feast-apply" Sep 08 08:39:36.932929 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:36.932815 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" containerName="feast-apply" Sep 08 08:39:36.932929 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:36.932821 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6bf283b-17f0-49a1-9782-96af409728d1" containerName="registry" Sep 08 08:39:36.932929 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:36.932826 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6bf283b-17f0-49a1-9782-96af409728d1" containerName="registry" Sep 08 08:39:36.932929 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:36.932833 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" containerName="online" Sep 08 08:39:36.932929 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:36.932838 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" containerName="online" Sep 08 08:39:36.932929 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:36.932848 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6bf283b-17f0-49a1-9782-96af409728d1" containerName="offline" Sep 08 08:39:36.932929 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:36.932854 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6bf283b-17f0-49a1-9782-96af409728d1" containerName="offline" Sep 08 08:39:36.932929 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:36.932860 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6bf283b-17f0-49a1-9782-96af409728d1" containerName="ui" Sep 08 08:39:36.932929 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:36.932865 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6bf283b-17f0-49a1-9782-96af409728d1" containerName="ui" Sep 08 08:39:36.932929 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:36.932876 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6bf283b-17f0-49a1-9782-96af409728d1" containerName="feast-apply" Sep 08 08:39:36.932929 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:36.932881 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6bf283b-17f0-49a1-9782-96af409728d1" containerName="feast-apply" Sep 08 08:39:36.932929 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:36.932923 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6bf283b-17f0-49a1-9782-96af409728d1" containerName="online" Sep 08 08:39:36.932929 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:36.932934 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6bf283b-17f0-49a1-9782-96af409728d1" containerName="registry" Sep 08 08:39:36.932929 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:36.932951 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6bf283b-17f0-49a1-9782-96af409728d1" containerName="ui" Sep 08 08:39:36.933976 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:36.932958 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" containerName="online" Sep 08 08:39:36.933976 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:36.932966 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" containerName="offline" Sep 08 08:39:36.933976 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:36.932971 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6bf283b-17f0-49a1-9782-96af409728d1" containerName="offline" Sep 08 08:39:36.933976 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:36.932976 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc9b3ea0-f3c5-41a7-8092-d9d49dc83be3" containerName="ui" Sep 08 08:39:36.938325 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:36.938292 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nqvlb/must-gather-qncfw" Sep 08 08:39:36.941196 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:36.941163 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nqvlb\"/\"kube-root-ca.crt\"" Sep 08 08:39:36.941966 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:36.941946 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nqvlb\"/\"openshift-service-ca.crt\"" Sep 08 08:39:36.942078 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:36.941973 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-nqvlb\"/\"default-dockercfg-gzzvd\"" Sep 08 08:39:36.943970 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:36.943949 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nqvlb/must-gather-qncfw"] Sep 08 08:39:37.052457 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:37.052432 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ef3e11e3-5417-4db0-95d4-86ea760500f7-must-gather-output\") pod \"must-gather-qncfw\" (UID: \"ef3e11e3-5417-4db0-95d4-86ea760500f7\") " pod="openshift-must-gather-nqvlb/must-gather-qncfw" Sep 08 08:39:37.052592 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:37.052524 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jl9kj\" (UniqueName: \"kubernetes.io/projected/ef3e11e3-5417-4db0-95d4-86ea760500f7-kube-api-access-jl9kj\") pod \"must-gather-qncfw\" (UID: \"ef3e11e3-5417-4db0-95d4-86ea760500f7\") " pod="openshift-must-gather-nqvlb/must-gather-qncfw" Sep 08 08:39:37.153652 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:37.153619 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl9kj\" (UniqueName: \"kubernetes.io/projected/ef3e11e3-5417-4db0-95d4-86ea760500f7-kube-api-access-jl9kj\") pod \"must-gather-qncfw\" (UID: \"ef3e11e3-5417-4db0-95d4-86ea760500f7\") " pod="openshift-must-gather-nqvlb/must-gather-qncfw" Sep 08 08:39:37.153652 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:37.153656 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ef3e11e3-5417-4db0-95d4-86ea760500f7-must-gather-output\") pod \"must-gather-qncfw\" (UID: \"ef3e11e3-5417-4db0-95d4-86ea760500f7\") " pod="openshift-must-gather-nqvlb/must-gather-qncfw" Sep 08 08:39:37.153935 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:37.153919 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ef3e11e3-5417-4db0-95d4-86ea760500f7-must-gather-output\") pod \"must-gather-qncfw\" (UID: \"ef3e11e3-5417-4db0-95d4-86ea760500f7\") " pod="openshift-must-gather-nqvlb/must-gather-qncfw" Sep 08 08:39:37.161899 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:37.161873 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl9kj\" (UniqueName: \"kubernetes.io/projected/ef3e11e3-5417-4db0-95d4-86ea760500f7-kube-api-access-jl9kj\") pod \"must-gather-qncfw\" (UID: \"ef3e11e3-5417-4db0-95d4-86ea760500f7\") " pod="openshift-must-gather-nqvlb/must-gather-qncfw" Sep 08 08:39:37.256962 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:37.256881 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nqvlb/must-gather-qncfw" Sep 08 08:39:37.530724 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:37.530648 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nqvlb/must-gather-qncfw"] Sep 08 08:39:37.533459 ip-10-0-143-1 kubenswrapper[2585]: W0908 08:39:37.533427 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podef3e11e3_5417_4db0_95d4_86ea760500f7.slice/crio-5522f0eac52f088429e825a7c9f978e87ee3b5e3e2c3e857d08c1407b7218407 WatchSource:0}: Error finding container 5522f0eac52f088429e825a7c9f978e87ee3b5e3e2c3e857d08c1407b7218407: Status 404 returned error can't find the container with id 5522f0eac52f088429e825a7c9f978e87ee3b5e3e2c3e857d08c1407b7218407 Sep 08 08:39:37.535202 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:37.535186 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 08 08:39:38.397742 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:38.397714 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nqvlb/must-gather-qncfw" event={"ID":"ef3e11e3-5417-4db0-95d4-86ea760500f7","Type":"ContainerStarted","Data":"5522f0eac52f088429e825a7c9f978e87ee3b5e3e2c3e857d08c1407b7218407"} Sep 08 08:39:39.403702 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:39.403660 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nqvlb/must-gather-qncfw" event={"ID":"ef3e11e3-5417-4db0-95d4-86ea760500f7","Type":"ContainerStarted","Data":"e97b615258ab2f9464aa678a8a235dfa2791d7068525522b07815e17ed60a63b"} Sep 08 08:39:39.404094 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:39.403717 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nqvlb/must-gather-qncfw" event={"ID":"ef3e11e3-5417-4db0-95d4-86ea760500f7","Type":"ContainerStarted","Data":"22365233ad336531a5019aab3e4316748c39eed41fef81e0fd30f94495bc9f20"} Sep 08 08:39:39.419184 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:39.419115 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nqvlb/must-gather-qncfw" podStartSLOduration=2.561571441 podStartE2EDuration="3.419094659s" totalImagesPullingTime="857.523218ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 08:39:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:39:39.417766668 +0000 UTC m=+1792.755406319" watchObservedRunningTime="2026-09-08 08:39:39.419094659 +0000 UTC m=+1792.756734301" Sep 08 08:39:39.733689 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:39.733609 2585 ???:1] "http: TLS handshake error from 10.0.130.42:35944: EOF" Sep 08 08:39:39.738817 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:39.738784 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-mczjr_dac5eaa3-6c21-4b05-ace5-a13c8344ab6e/global-pull-secret-syncer/0.log" Sep 08 08:39:39.890324 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:39.890288 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ffdd8_fbc11a97-76be-4f9f-a3d1-354de4d4f1b5/konnectivity-agent/0.log" Sep 08 08:39:39.964129 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:39.964099 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-1.ec2.internal_6f38569e02cf6895d157f6aa28cad740/haproxy/0.log" Sep 08 08:39:43.697753 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:43.697670 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7932f481-a24e-4554-9401-b69cf440085e/alertmanager/0.log" Sep 08 08:39:43.727288 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:43.727256 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7932f481-a24e-4554-9401-b69cf440085e/config-reloader/0.log" Sep 08 08:39:43.752461 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:43.752435 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7932f481-a24e-4554-9401-b69cf440085e/kube-rbac-proxy-web/0.log" Sep 08 08:39:43.774888 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:43.774812 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7932f481-a24e-4554-9401-b69cf440085e/kube-rbac-proxy/0.log" Sep 08 08:39:43.798038 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:43.798002 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7932f481-a24e-4554-9401-b69cf440085e/kube-rbac-proxy-metric/0.log" Sep 08 08:39:43.823705 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:43.823675 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7932f481-a24e-4554-9401-b69cf440085e/prom-label-proxy/0.log" Sep 08 08:39:43.846627 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:43.846539 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7932f481-a24e-4554-9401-b69cf440085e/init-config-reloader/0.log" Sep 08 08:39:44.029566 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:44.029531 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-5ncz8_69650628-eaa4-400d-9c77-eacece44749a/monitoring-plugin/0.log" Sep 08 08:39:44.058771 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:44.058737 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-j5m5d_ecca6e11-9831-4fe4-9e0d-aacadb1a404c/node-exporter/0.log" Sep 08 08:39:44.082268 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:44.082235 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-j5m5d_ecca6e11-9831-4fe4-9e0d-aacadb1a404c/kube-rbac-proxy/0.log" Sep 08 08:39:44.105654 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:44.105409 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-j5m5d_ecca6e11-9831-4fe4-9e0d-aacadb1a404c/init-textfile/0.log" Sep 08 08:39:44.305745 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:44.305717 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-8qxcv_45b90f7c-de16-4a7f-b5ab-9cd28caa701c/kube-rbac-proxy-main/0.log" Sep 08 08:39:44.332883 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:44.332795 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-8qxcv_45b90f7c-de16-4a7f-b5ab-9cd28caa701c/kube-rbac-proxy-self/0.log" Sep 08 08:39:44.358904 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:44.358826 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-8qxcv_45b90f7c-de16-4a7f-b5ab-9cd28caa701c/openshift-state-metrics/0.log" Sep 08 08:39:44.396051 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:44.396019 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_581c35a0-bb28-4749-a432-8d4757b0abcf/prometheus/0.log" Sep 08 08:39:44.417281 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:44.417253 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_581c35a0-bb28-4749-a432-8d4757b0abcf/config-reloader/0.log" Sep 08 08:39:44.439776 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:44.439749 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_581c35a0-bb28-4749-a432-8d4757b0abcf/thanos-sidecar/0.log" Sep 08 08:39:44.464044 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:44.464013 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_581c35a0-bb28-4749-a432-8d4757b0abcf/kube-rbac-proxy-web/0.log" Sep 08 08:39:44.485402 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:44.485363 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_581c35a0-bb28-4749-a432-8d4757b0abcf/kube-rbac-proxy/0.log" Sep 08 08:39:44.508396 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:44.508355 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_581c35a0-bb28-4749-a432-8d4757b0abcf/kube-rbac-proxy-thanos/0.log" Sep 08 08:39:44.530137 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:44.530106 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_581c35a0-bb28-4749-a432-8d4757b0abcf/init-config-reloader/0.log" Sep 08 08:39:44.559295 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:44.559259 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-tvhls_6333538e-9193-4692-946f-8f44c4869ceb/prometheus-operator/0.log" Sep 08 08:39:44.585089 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:44.585042 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-tvhls_6333538e-9193-4692-946f-8f44c4869ceb/kube-rbac-proxy/0.log" Sep 08 08:39:44.735333 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:44.735301 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7699cb4d56-j2wp8_edebb973-5c2f-4951-971b-eb30724528c9/thanos-query/0.log" Sep 08 08:39:44.764129 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:44.764089 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7699cb4d56-j2wp8_edebb973-5c2f-4951-971b-eb30724528c9/kube-rbac-proxy-web/0.log" Sep 08 08:39:44.787877 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:44.787839 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7699cb4d56-j2wp8_edebb973-5c2f-4951-971b-eb30724528c9/kube-rbac-proxy/0.log" Sep 08 08:39:44.811020 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:44.810992 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7699cb4d56-j2wp8_edebb973-5c2f-4951-971b-eb30724528c9/prom-label-proxy/0.log" Sep 08 08:39:44.836590 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:44.836558 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7699cb4d56-j2wp8_edebb973-5c2f-4951-971b-eb30724528c9/kube-rbac-proxy-rules/0.log" Sep 08 08:39:44.866915 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:44.866845 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7699cb4d56-j2wp8_edebb973-5c2f-4951-971b-eb30724528c9/kube-rbac-proxy-metrics/0.log" Sep 08 08:39:47.208635 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:47.208610 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2xlvk_e00cae51-f771-4039-81ec-54fa2a48e98f/ovn-acl-logging/0.log" Sep 08 08:39:47.210920 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:47.210895 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2xlvk_e00cae51-f771-4039-81ec-54fa2a48e98f/ovn-acl-logging/0.log" Sep 08 08:39:47.345885 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:47.345847 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nqvlb/perf-node-gather-daemonset-twpld"] Sep 08 08:39:47.352276 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:47.352243 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nqvlb/perf-node-gather-daemonset-twpld" Sep 08 08:39:47.361185 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:47.361153 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nqvlb/perf-node-gather-daemonset-twpld"] Sep 08 08:39:47.445580 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:47.445536 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nt7j7\" (UniqueName: \"kubernetes.io/projected/9fa9000a-f4bd-4fdd-b521-bf9c545883f3-kube-api-access-nt7j7\") pod \"perf-node-gather-daemonset-twpld\" (UID: \"9fa9000a-f4bd-4fdd-b521-bf9c545883f3\") " pod="openshift-must-gather-nqvlb/perf-node-gather-daemonset-twpld" Sep 08 08:39:47.445767 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:47.445618 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9fa9000a-f4bd-4fdd-b521-bf9c545883f3-proc\") pod \"perf-node-gather-daemonset-twpld\" (UID: \"9fa9000a-f4bd-4fdd-b521-bf9c545883f3\") " pod="openshift-must-gather-nqvlb/perf-node-gather-daemonset-twpld" Sep 08 08:39:47.445767 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:47.445643 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9fa9000a-f4bd-4fdd-b521-bf9c545883f3-lib-modules\") pod \"perf-node-gather-daemonset-twpld\" (UID: \"9fa9000a-f4bd-4fdd-b521-bf9c545883f3\") " pod="openshift-must-gather-nqvlb/perf-node-gather-daemonset-twpld" Sep 08 08:39:47.445767 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:47.445701 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9fa9000a-f4bd-4fdd-b521-bf9c545883f3-sys\") pod \"perf-node-gather-daemonset-twpld\" (UID: \"9fa9000a-f4bd-4fdd-b521-bf9c545883f3\") " pod="openshift-must-gather-nqvlb/perf-node-gather-daemonset-twpld" Sep 08 08:39:47.445767 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:47.445729 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9fa9000a-f4bd-4fdd-b521-bf9c545883f3-podres\") pod \"perf-node-gather-daemonset-twpld\" (UID: \"9fa9000a-f4bd-4fdd-b521-bf9c545883f3\") " pod="openshift-must-gather-nqvlb/perf-node-gather-daemonset-twpld" Sep 08 08:39:47.546530 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:47.546444 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nt7j7\" (UniqueName: \"kubernetes.io/projected/9fa9000a-f4bd-4fdd-b521-bf9c545883f3-kube-api-access-nt7j7\") pod \"perf-node-gather-daemonset-twpld\" (UID: \"9fa9000a-f4bd-4fdd-b521-bf9c545883f3\") " pod="openshift-must-gather-nqvlb/perf-node-gather-daemonset-twpld" Sep 08 08:39:47.546792 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:47.546776 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9fa9000a-f4bd-4fdd-b521-bf9c545883f3-proc\") pod \"perf-node-gather-daemonset-twpld\" (UID: \"9fa9000a-f4bd-4fdd-b521-bf9c545883f3\") " pod="openshift-must-gather-nqvlb/perf-node-gather-daemonset-twpld" Sep 08 08:39:47.546986 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:47.546972 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9fa9000a-f4bd-4fdd-b521-bf9c545883f3-lib-modules\") pod \"perf-node-gather-daemonset-twpld\" (UID: \"9fa9000a-f4bd-4fdd-b521-bf9c545883f3\") " pod="openshift-must-gather-nqvlb/perf-node-gather-daemonset-twpld" Sep 08 08:39:47.547292 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:47.547267 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9fa9000a-f4bd-4fdd-b521-bf9c545883f3-sys\") pod \"perf-node-gather-daemonset-twpld\" (UID: \"9fa9000a-f4bd-4fdd-b521-bf9c545883f3\") " pod="openshift-must-gather-nqvlb/perf-node-gather-daemonset-twpld" Sep 08 08:39:47.547426 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:47.547310 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9fa9000a-f4bd-4fdd-b521-bf9c545883f3-podres\") pod \"perf-node-gather-daemonset-twpld\" (UID: \"9fa9000a-f4bd-4fdd-b521-bf9c545883f3\") " pod="openshift-must-gather-nqvlb/perf-node-gather-daemonset-twpld" Sep 08 08:39:47.547426 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:47.546931 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9fa9000a-f4bd-4fdd-b521-bf9c545883f3-proc\") pod \"perf-node-gather-daemonset-twpld\" (UID: \"9fa9000a-f4bd-4fdd-b521-bf9c545883f3\") " pod="openshift-must-gather-nqvlb/perf-node-gather-daemonset-twpld" Sep 08 08:39:47.547426 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:47.547177 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9fa9000a-f4bd-4fdd-b521-bf9c545883f3-lib-modules\") pod \"perf-node-gather-daemonset-twpld\" (UID: \"9fa9000a-f4bd-4fdd-b521-bf9c545883f3\") " pod="openshift-must-gather-nqvlb/perf-node-gather-daemonset-twpld" Sep 08 08:39:47.547426 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:47.547363 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9fa9000a-f4bd-4fdd-b521-bf9c545883f3-sys\") pod \"perf-node-gather-daemonset-twpld\" (UID: \"9fa9000a-f4bd-4fdd-b521-bf9c545883f3\") " pod="openshift-must-gather-nqvlb/perf-node-gather-daemonset-twpld" Sep 08 08:39:47.547583 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:47.547480 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9fa9000a-f4bd-4fdd-b521-bf9c545883f3-podres\") pod \"perf-node-gather-daemonset-twpld\" (UID: \"9fa9000a-f4bd-4fdd-b521-bf9c545883f3\") " pod="openshift-must-gather-nqvlb/perf-node-gather-daemonset-twpld" Sep 08 08:39:47.556158 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:47.556126 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nt7j7\" (UniqueName: \"kubernetes.io/projected/9fa9000a-f4bd-4fdd-b521-bf9c545883f3-kube-api-access-nt7j7\") pod \"perf-node-gather-daemonset-twpld\" (UID: \"9fa9000a-f4bd-4fdd-b521-bf9c545883f3\") " pod="openshift-must-gather-nqvlb/perf-node-gather-daemonset-twpld" Sep 08 08:39:47.665634 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:47.665604 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nqvlb/perf-node-gather-daemonset-twpld" Sep 08 08:39:47.937607 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:47.937557 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nqvlb/perf-node-gather-daemonset-twpld"] Sep 08 08:39:48.433819 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:48.433794 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rdr6d_9f709c93-d0b7-4369-98aa-5e912759dead/dns/0.log" Sep 08 08:39:48.453269 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:48.453248 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rdr6d_9f709c93-d0b7-4369-98aa-5e912759dead/kube-rbac-proxy/0.log" Sep 08 08:39:48.461453 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:48.461425 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nqvlb/perf-node-gather-daemonset-twpld" event={"ID":"9fa9000a-f4bd-4fdd-b521-bf9c545883f3","Type":"ContainerStarted","Data":"28c254818289cf6f9566f31e52cefef41ce513694e594b8fb699427323472696"} Sep 08 08:39:48.461597 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:48.461473 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nqvlb/perf-node-gather-daemonset-twpld" event={"ID":"9fa9000a-f4bd-4fdd-b521-bf9c545883f3","Type":"ContainerStarted","Data":"42cc6396e7e69457a3f250b617aeb07028c7adef744fdc00cef147bdee1bb63a"} Sep 08 08:39:48.461597 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:48.461499 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-nqvlb/perf-node-gather-daemonset-twpld" Sep 08 08:39:48.477927 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:48.477889 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bxkxt_6eee659d-776f-4175-9e5a-6574fdd0e06c/dns-node-resolver/0.log" Sep 08 08:39:48.478629 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:48.478578 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nqvlb/perf-node-gather-daemonset-twpld" podStartSLOduration=1.47856012 podStartE2EDuration="1.47856012s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 08:39:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:39:48.477015682 +0000 UTC m=+1801.814655324" watchObservedRunningTime="2026-09-08 08:39:48.47856012 +0000 UTC m=+1801.816199762" Sep 08 08:39:49.008577 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:49.008551 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jqfnk_6698964c-4407-4737-a7ca-c673a3feef82/node-ca/0.log" Sep 08 08:39:50.034555 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:50.034521 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jbjgx_f0afdfcb-a4d0-41e1-8928-f251568daaef/serve-healthcheck-canary/0.log" Sep 08 08:39:50.483148 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:50.483105 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v5pd2_8fd236ec-d3a6-4c13-b01f-6035a2d96c55/kube-rbac-proxy/0.log" Sep 08 08:39:50.502701 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:50.502676 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v5pd2_8fd236ec-d3a6-4c13-b01f-6035a2d96c55/exporter/0.log" Sep 08 08:39:50.522003 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:50.521974 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v5pd2_8fd236ec-d3a6-4c13-b01f-6035a2d96c55/extractor/0.log" Sep 08 08:39:54.478577 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:54.478547 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-nqvlb/perf-node-gather-daemonset-twpld" Sep 08 08:39:55.933612 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:55.933578 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-2r8ll_f66a9794-3b59-486b-ba0a-348a31d3b263/kube-multus/0.log" Sep 08 08:39:56.348814 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:56.348784 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zwkjc_fcd3e62c-7a33-4211-9b1f-1223aeb19d09/kube-multus-additional-cni-plugins/0.log" Sep 08 08:39:56.374581 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:56.374547 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zwkjc_fcd3e62c-7a33-4211-9b1f-1223aeb19d09/egress-router-binary-copy/0.log" Sep 08 08:39:56.404115 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:56.404071 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zwkjc_fcd3e62c-7a33-4211-9b1f-1223aeb19d09/cni-plugins/0.log" Sep 08 08:39:56.428554 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:56.428523 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zwkjc_fcd3e62c-7a33-4211-9b1f-1223aeb19d09/bond-cni-plugin/0.log" Sep 08 08:39:56.454495 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:56.454444 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zwkjc_fcd3e62c-7a33-4211-9b1f-1223aeb19d09/routeoverride-cni/0.log" Sep 08 08:39:56.477602 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:56.477570 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zwkjc_fcd3e62c-7a33-4211-9b1f-1223aeb19d09/whereabouts-cni-bincopy/0.log" Sep 08 08:39:56.499754 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:56.499674 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zwkjc_fcd3e62c-7a33-4211-9b1f-1223aeb19d09/whereabouts-cni/0.log" Sep 08 08:39:56.613467 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:56.613441 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ns5tq_95940b17-787c-4892-96f0-83cf9501e4d5/network-metrics-daemon/0.log" Sep 08 08:39:56.633965 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:56.633940 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ns5tq_95940b17-787c-4892-96f0-83cf9501e4d5/kube-rbac-proxy/0.log" Sep 08 08:39:57.433941 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:57.433909 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2xlvk_e00cae51-f771-4039-81ec-54fa2a48e98f/ovn-controller/0.log" Sep 08 08:39:57.452440 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:57.452412 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2xlvk_e00cae51-f771-4039-81ec-54fa2a48e98f/ovn-acl-logging/0.log" Sep 08 08:39:57.452440 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:57.452445 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2xlvk_e00cae51-f771-4039-81ec-54fa2a48e98f/ovn-acl-logging/1.log" Sep 08 08:39:57.469522 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:57.469497 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2xlvk_e00cae51-f771-4039-81ec-54fa2a48e98f/kube-rbac-proxy-node/0.log" Sep 08 08:39:57.489105 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:57.489077 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2xlvk_e00cae51-f771-4039-81ec-54fa2a48e98f/kube-rbac-proxy-ovn-metrics/0.log" Sep 08 08:39:57.508469 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:57.508448 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2xlvk_e00cae51-f771-4039-81ec-54fa2a48e98f/northd/0.log" Sep 08 08:39:57.529741 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:57.529721 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2xlvk_e00cae51-f771-4039-81ec-54fa2a48e98f/nbdb/0.log" Sep 08 08:39:57.548954 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:57.548918 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2xlvk_e00cae51-f771-4039-81ec-54fa2a48e98f/sbdb/0.log" Sep 08 08:39:57.592582 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:57.592552 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2xlvk_e00cae51-f771-4039-81ec-54fa2a48e98f/ovnkube-controller/0.log" Sep 08 08:39:58.967240 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:58.967205 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-k45rp_7301238b-b7d9-4101-8f7a-5e442499ca05/network-check-target-container/0.log" Sep 08 08:39:59.892949 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:39:59.892914 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-hhv6k_f4583a67-e97f-467b-a658-fe2d36c197e2/iptables-alerter/0.log" Sep 08 08:40:00.518962 ip-10-0-143-1 kubenswrapper[2585]: I0908 08:40:00.518930 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-hnpc6_9852555a-bf3c-4370-b542-d139f523ebf0/tuned/0.log"