Sep 08 08:45:42.574526 ip-10-0-141-177 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 08 08:45:42.574537 ip-10-0-141-177 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 08 08:45:42.574544 ip-10-0-141-177 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 08 08:45:42.574679 ip-10-0-141-177 systemd[1]: Failed to start Kubernetes Kubelet. Sep 08 08:45:52.774017 ip-10-0-141-177 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 08 08:45:52.774035 ip-10-0-141-177 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot d208a7d5bacc44b384ffe78d5777b532 -- Sep 08 08:47:32.145358 ip-10-0-141-177 systemd[1]: Starting Kubernetes Kubelet... Sep 08 08:47:32.553821 ip-10-0-141-177 kubenswrapper[2554]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 08 08:47:32.553821 ip-10-0-141-177 kubenswrapper[2554]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 08 08:47:32.553821 ip-10-0-141-177 kubenswrapper[2554]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 08 08:47:32.553821 ip-10-0-141-177 kubenswrapper[2554]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 08 08:47:32.553821 ip-10-0-141-177 kubenswrapper[2554]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 08 08:47:32.554682 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.554601 2554 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 08 08:47:32.557653 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557639 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 08:47:32.557653 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557653 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 08:47:32.557715 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557657 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 08:47:32.557715 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557661 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 08:47:32.557715 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557664 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 08:47:32.557715 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557667 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 08:47:32.557715 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557671 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 08:47:32.557715 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557674 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 08:47:32.557715 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557676 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 08:47:32.557715 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557679 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 08:47:32.557715 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557682 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 08:47:32.557715 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557684 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 08:47:32.557715 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557687 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 08:47:32.557715 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557689 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 08:47:32.557715 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557694 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 08:47:32.557715 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557697 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 08:47:32.557715 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557700 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 08:47:32.557715 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557702 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 08:47:32.557715 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557705 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 08:47:32.557715 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557708 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 08:47:32.557715 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557711 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 08:47:32.557715 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557713 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 08:47:32.558199 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557716 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 08:47:32.558199 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557719 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 08:47:32.558199 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557722 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 08:47:32.558199 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557725 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 08:47:32.558199 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557728 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 08:47:32.558199 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557731 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 08:47:32.558199 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557733 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 08:47:32.558199 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557736 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 08:47:32.558199 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557738 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 08:47:32.558199 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557741 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 08:47:32.558199 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557743 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 08:47:32.558199 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557746 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 08:47:32.558199 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557749 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 08:47:32.558199 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557752 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 08:47:32.558199 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557754 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 08:47:32.558199 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557757 2554 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 08:47:32.558199 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557759 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 08:47:32.558199 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557762 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 08:47:32.558199 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557766 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 08:47:32.558199 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557770 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 08:47:32.558723 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557772 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 08:47:32.558723 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557775 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 08:47:32.558723 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557778 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 08:47:32.558723 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557781 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 08:47:32.558723 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557784 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 08:47:32.558723 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557787 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 08:47:32.558723 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557789 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 08:47:32.558723 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557792 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 08:47:32.558723 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557797 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 08:47:32.558723 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557800 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 08:47:32.558723 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557803 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 08:47:32.558723 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557806 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 08:47:32.558723 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557809 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 08:47:32.558723 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557813 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 08:47:32.558723 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557816 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 08:47:32.558723 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557819 2554 feature_gate.go:328] unrecognized feature gate: Example Sep 08 08:47:32.558723 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557822 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 08:47:32.558723 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557825 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 08:47:32.558723 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557828 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 08:47:32.558723 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557830 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 08:47:32.559231 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557833 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 08:47:32.559231 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557836 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 08:47:32.559231 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557838 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 08:47:32.559231 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557841 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 08:47:32.559231 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557844 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 08:47:32.559231 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557846 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 08:47:32.559231 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557848 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 08:47:32.559231 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557851 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 08:47:32.559231 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557853 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 08:47:32.559231 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557856 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 08:47:32.559231 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557859 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 08:47:32.559231 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557862 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 08:47:32.559231 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557865 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 08:47:32.559231 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557867 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 08:47:32.559231 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557871 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 08:47:32.559231 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557874 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 08:47:32.559231 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557876 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 08:47:32.559231 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557879 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 08:47:32.559231 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557881 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 08:47:32.559696 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557884 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 08:47:32.559696 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557886 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 08:47:32.559696 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557889 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 08:47:32.559696 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557892 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 08:47:32.559696 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557894 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 08:47:32.559696 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557897 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 08:47:32.559696 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557900 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 08:47:32.559696 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557903 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 08:47:32.559696 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557905 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 08:47:32.559696 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557909 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 08:47:32.559696 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557911 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 08:47:32.559696 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557914 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 08:47:32.559696 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557916 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 08:47:32.559696 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557918 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 08:47:32.559696 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557921 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 08:47:32.559696 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557923 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 08:47:32.559696 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557926 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 08:47:32.559696 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.557928 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 08:47:32.559696 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558606 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 08:47:32.559696 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558612 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 08:47:32.560192 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558616 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 08:47:32.560192 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558618 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 08:47:32.560192 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558621 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 08:47:32.560192 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558624 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 08:47:32.560192 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558627 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 08:47:32.560192 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558630 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 08:47:32.560192 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558634 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 08:47:32.560192 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558638 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 08:47:32.560192 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558641 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 08:47:32.560192 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558644 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 08:47:32.560192 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558647 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 08:47:32.560192 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558650 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 08:47:32.560192 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558653 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 08:47:32.560192 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558656 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 08:47:32.560192 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558658 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 08:47:32.560192 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558661 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 08:47:32.560192 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558664 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 08:47:32.560192 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558667 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 08:47:32.560192 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558671 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 08:47:32.560660 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558673 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 08:47:32.560660 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558676 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 08:47:32.560660 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558679 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 08:47:32.560660 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558682 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 08:47:32.560660 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558684 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 08:47:32.560660 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558687 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 08:47:32.560660 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558690 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 08:47:32.560660 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558693 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 08:47:32.560660 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558695 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 08:47:32.560660 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558698 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 08:47:32.560660 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558700 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 08:47:32.560660 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558703 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 08:47:32.560660 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558707 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 08:47:32.560660 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558710 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 08:47:32.560660 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558713 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 08:47:32.560660 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558716 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 08:47:32.560660 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558719 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 08:47:32.560660 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558722 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 08:47:32.560660 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558725 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 08:47:32.560660 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558728 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 08:47:32.561183 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558730 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 08:47:32.561183 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558733 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 08:47:32.561183 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558735 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 08:47:32.561183 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558738 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 08:47:32.561183 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558741 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 08:47:32.561183 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558743 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 08:47:32.561183 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558746 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 08:47:32.561183 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558748 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 08:47:32.561183 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558751 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 08:47:32.561183 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558754 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 08:47:32.561183 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558756 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 08:47:32.561183 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558759 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 08:47:32.561183 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558762 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 08:47:32.561183 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558764 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 08:47:32.561183 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558767 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 08:47:32.561183 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558770 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 08:47:32.561183 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558772 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 08:47:32.561183 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558775 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 08:47:32.561183 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558777 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 08:47:32.561183 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558780 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 08:47:32.561669 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558782 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 08:47:32.561669 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558784 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 08:47:32.561669 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558787 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 08:47:32.561669 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558790 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 08:47:32.561669 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558792 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 08:47:32.561669 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558795 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 08:47:32.561669 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558797 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 08:47:32.561669 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558800 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 08:47:32.561669 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558803 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 08:47:32.561669 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558805 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 08:47:32.561669 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558808 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 08:47:32.561669 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558811 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 08:47:32.561669 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558814 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 08:47:32.561669 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558818 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 08:47:32.561669 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558821 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 08:47:32.561669 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558823 2554 feature_gate.go:328] unrecognized feature gate: Example Sep 08 08:47:32.561669 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558826 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 08:47:32.561669 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558828 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 08:47:32.561669 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558831 2554 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 08:47:32.561669 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558833 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 08:47:32.562163 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558836 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 08:47:32.562163 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558838 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 08:47:32.562163 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558841 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 08:47:32.562163 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558844 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 08:47:32.562163 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558847 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 08:47:32.562163 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558850 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 08:47:32.562163 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558853 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 08:47:32.562163 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558855 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 08:47:32.562163 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558858 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 08:47:32.562163 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558862 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 08:47:32.562163 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558866 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 08:47:32.562163 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558868 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 08:47:32.562163 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558871 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 08:47:32.562163 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558874 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 08:47:32.562163 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558876 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 08:47:32.562163 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558879 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 08:47:32.562163 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558882 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 08:47:32.562163 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.558884 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 08:47:32.562163 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559239 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 08:47:32.562633 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559244 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 08:47:32.562633 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559248 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 08:47:32.562633 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559251 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 08:47:32.562633 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559253 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 08:47:32.562633 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559256 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 08:47:32.562633 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559259 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 08:47:32.562633 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559262 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 08:47:32.562633 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559265 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 08:47:32.562633 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559267 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 08:47:32.562633 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559271 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 08:47:32.562633 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559275 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 08:47:32.562633 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559279 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 08:47:32.562633 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559282 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 08:47:32.562633 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559285 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 08:47:32.562633 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559288 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 08:47:32.562633 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559291 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 08:47:32.562633 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559294 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 08:47:32.562633 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559297 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 08:47:32.562633 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559300 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 08:47:32.563114 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559303 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 08:47:32.563114 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559306 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 08:47:32.563114 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559309 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 08:47:32.563114 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559313 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 08:47:32.563114 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559316 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 08:47:32.563114 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559318 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 08:47:32.563114 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559321 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 08:47:32.563114 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559323 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 08:47:32.563114 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559327 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 08:47:32.563114 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559330 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 08:47:32.563114 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559333 2554 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 08:47:32.563114 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559336 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 08:47:32.563114 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559339 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 08:47:32.563114 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559342 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 08:47:32.563114 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559344 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 08:47:32.563114 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559347 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 08:47:32.563114 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559349 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 08:47:32.563114 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559352 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 08:47:32.563114 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559355 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 08:47:32.563114 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559358 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 08:47:32.563605 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559361 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 08:47:32.563605 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559363 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 08:47:32.563605 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559366 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 08:47:32.563605 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559369 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 08:47:32.563605 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559371 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 08:47:32.563605 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559373 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 08:47:32.563605 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559376 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 08:47:32.563605 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559378 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 08:47:32.563605 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559381 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 08:47:32.563605 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559384 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 08:47:32.563605 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559387 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 08:47:32.563605 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559389 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 08:47:32.563605 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559392 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 08:47:32.563605 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559394 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 08:47:32.563605 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559397 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 08:47:32.563605 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559399 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 08:47:32.563605 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559403 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 08:47:32.563605 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559406 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 08:47:32.563605 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559410 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 08:47:32.564087 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559412 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 08:47:32.564087 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559415 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 08:47:32.564087 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559418 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 08:47:32.564087 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559421 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 08:47:32.564087 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559424 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 08:47:32.564087 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559427 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 08:47:32.564087 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559429 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 08:47:32.564087 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559432 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 08:47:32.564087 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559435 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 08:47:32.564087 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559437 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 08:47:32.564087 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559440 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 08:47:32.564087 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559443 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 08:47:32.564087 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559446 2554 feature_gate.go:328] unrecognized feature gate: Example Sep 08 08:47:32.564087 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559448 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 08:47:32.564087 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559451 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 08:47:32.564087 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559453 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 08:47:32.564087 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559456 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 08:47:32.564087 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559458 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 08:47:32.564087 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559461 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 08:47:32.564087 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559463 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 08:47:32.564087 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559467 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 08:47:32.564605 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559469 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 08:47:32.564605 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559472 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 08:47:32.564605 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559474 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 08:47:32.564605 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559477 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 08:47:32.564605 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559480 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 08:47:32.564605 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559482 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 08:47:32.564605 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559485 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 08:47:32.564605 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559487 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 08:47:32.564605 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559490 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 08:47:32.564605 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559493 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 08:47:32.564605 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559495 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 08:47:32.564605 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559498 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 08:47:32.564605 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559500 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 08:47:32.564605 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559503 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 08:47:32.564605 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559506 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 08:47:32.564605 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559508 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 08:47:32.564605 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559511 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 08:47:32.564605 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559513 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 08:47:32.564605 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.559516 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 08:47:32.564605 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559584 2554 flags.go:64] FLAG: --address="0.0.0.0" Sep 08 08:47:32.565108 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559616 2554 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 08 08:47:32.565108 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559623 2554 flags.go:64] FLAG: --anonymous-auth="true" Sep 08 08:47:32.565108 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559628 2554 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 08 08:47:32.565108 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559632 2554 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 08 08:47:32.565108 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559638 2554 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 08 08:47:32.565108 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559642 2554 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 08 08:47:32.565108 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559647 2554 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 08 08:47:32.565108 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559650 2554 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 08 08:47:32.565108 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559652 2554 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 08 08:47:32.565108 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559656 2554 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 08 08:47:32.565108 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559659 2554 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 08 08:47:32.565108 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559662 2554 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 08 08:47:32.565108 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559665 2554 flags.go:64] FLAG: --cgroup-root="" Sep 08 08:47:32.565108 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559669 2554 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 08 08:47:32.565108 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559672 2554 flags.go:64] FLAG: --client-ca-file="" Sep 08 08:47:32.565108 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559675 2554 flags.go:64] FLAG: --cloud-provider="external" Sep 08 08:47:32.565108 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559678 2554 flags.go:64] FLAG: --cluster-dns="[]" Sep 08 08:47:32.565108 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559682 2554 flags.go:64] FLAG: --cluster-domain="" Sep 08 08:47:32.565108 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559684 2554 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 08 08:47:32.565108 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559687 2554 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 08 08:47:32.565108 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559690 2554 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 08 08:47:32.565108 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559694 2554 flags.go:64] FLAG: --container-log-max-files="5" Sep 08 08:47:32.565108 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559698 2554 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 08 08:47:32.565108 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559700 2554 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 08 08:47:32.565680 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559703 2554 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 08 08:47:32.565680 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559707 2554 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 08 08:47:32.565680 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559710 2554 flags.go:64] FLAG: --contention-profiling="false" Sep 08 08:47:32.565680 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559713 2554 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 08 08:47:32.565680 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559716 2554 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 08 08:47:32.565680 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559719 2554 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 08 08:47:32.565680 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559722 2554 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 08 08:47:32.565680 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559726 2554 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 08 08:47:32.565680 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559729 2554 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 08 08:47:32.565680 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559732 2554 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 08 08:47:32.565680 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559737 2554 flags.go:64] FLAG: --enable-load-reader="false" Sep 08 08:47:32.565680 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559740 2554 flags.go:64] FLAG: --enable-server="true" Sep 08 08:47:32.565680 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559743 2554 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 08 08:47:32.565680 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559748 2554 flags.go:64] FLAG: --event-burst="100" Sep 08 08:47:32.565680 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559751 2554 flags.go:64] FLAG: --event-qps="50" Sep 08 08:47:32.565680 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559754 2554 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 08 08:47:32.565680 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559757 2554 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 08 08:47:32.565680 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559759 2554 flags.go:64] FLAG: --eviction-hard="" Sep 08 08:47:32.565680 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559763 2554 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 08 08:47:32.565680 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559766 2554 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 08 08:47:32.565680 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559768 2554 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 08 08:47:32.565680 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559772 2554 flags.go:64] FLAG: --eviction-soft="" Sep 08 08:47:32.565680 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559774 2554 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 08 08:47:32.565680 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559777 2554 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 08 08:47:32.565680 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559780 2554 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 08 08:47:32.566348 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559783 2554 flags.go:64] FLAG: --experimental-mounter-path="" Sep 08 08:47:32.566348 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559786 2554 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 08 08:47:32.566348 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559789 2554 flags.go:64] FLAG: --fail-swap-on="true" Sep 08 08:47:32.566348 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559792 2554 flags.go:64] FLAG: --feature-gates="" Sep 08 08:47:32.566348 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559795 2554 flags.go:64] FLAG: --file-check-frequency="20s" Sep 08 08:47:32.566348 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559798 2554 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 08 08:47:32.566348 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559801 2554 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 08 08:47:32.566348 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559804 2554 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 08 08:47:32.566348 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559807 2554 flags.go:64] FLAG: --healthz-port="10248" Sep 08 08:47:32.566348 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559810 2554 flags.go:64] FLAG: --help="false" Sep 08 08:47:32.566348 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559813 2554 flags.go:64] FLAG: --hostname-override="ip-10-0-141-177.ec2.internal" Sep 08 08:47:32.566348 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559816 2554 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 08 08:47:32.566348 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559819 2554 flags.go:64] FLAG: --http-check-frequency="20s" Sep 08 08:47:32.566348 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559822 2554 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 08 08:47:32.566348 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559825 2554 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 08 08:47:32.566348 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559828 2554 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 08 08:47:32.566348 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559831 2554 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 08 08:47:32.566348 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559835 2554 flags.go:64] FLAG: --image-service-endpoint="" Sep 08 08:47:32.566348 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559838 2554 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 08 08:47:32.566348 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559840 2554 flags.go:64] FLAG: --kube-api-burst="100" Sep 08 08:47:32.566348 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559843 2554 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 08 08:47:32.566348 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559846 2554 flags.go:64] FLAG: --kube-api-qps="50" Sep 08 08:47:32.566348 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559849 2554 flags.go:64] FLAG: --kube-reserved="" Sep 08 08:47:32.566348 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559852 2554 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 08 08:47:32.566985 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559855 2554 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 08 08:47:32.566985 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559858 2554 flags.go:64] FLAG: --kubelet-cgroups="" Sep 08 08:47:32.566985 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559860 2554 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 08 08:47:32.566985 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559863 2554 flags.go:64] FLAG: --lock-file="" Sep 08 08:47:32.566985 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559867 2554 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 08 08:47:32.566985 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559870 2554 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 08 08:47:32.566985 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559873 2554 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 08 08:47:32.566985 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559878 2554 flags.go:64] FLAG: --log-json-split-stream="false" Sep 08 08:47:32.566985 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559881 2554 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 08 08:47:32.566985 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559884 2554 flags.go:64] FLAG: --log-text-split-stream="false" Sep 08 08:47:32.566985 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559886 2554 flags.go:64] FLAG: --logging-format="text" Sep 08 08:47:32.566985 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559889 2554 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 08 08:47:32.566985 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559892 2554 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 08 08:47:32.566985 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559895 2554 flags.go:64] FLAG: --manifest-url="" Sep 08 08:47:32.566985 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559898 2554 flags.go:64] FLAG: --manifest-url-header="" Sep 08 08:47:32.566985 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559902 2554 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 08 08:47:32.566985 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559905 2554 flags.go:64] FLAG: --max-open-files="1000000" Sep 08 08:47:32.566985 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559909 2554 flags.go:64] FLAG: --max-pods="110" Sep 08 08:47:32.566985 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559912 2554 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 08 08:47:32.566985 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559914 2554 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 08 08:47:32.566985 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559917 2554 flags.go:64] FLAG: --memory-manager-policy="None" Sep 08 08:47:32.566985 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559920 2554 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 08 08:47:32.566985 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559923 2554 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 08 08:47:32.566985 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559926 2554 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 08 08:47:32.566985 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559929 2554 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 08 08:47:32.568012 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559937 2554 flags.go:64] FLAG: --node-status-max-images="50" Sep 08 08:47:32.568012 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559939 2554 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 08 08:47:32.568012 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559942 2554 flags.go:64] FLAG: --oom-score-adj="-999" Sep 08 08:47:32.568012 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559945 2554 flags.go:64] FLAG: --pod-cidr="" Sep 08 08:47:32.568012 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559948 2554 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 08 08:47:32.568012 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559954 2554 flags.go:64] FLAG: --pod-manifest-path="" Sep 08 08:47:32.568012 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559957 2554 flags.go:64] FLAG: --pod-max-pids="-1" Sep 08 08:47:32.568012 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559960 2554 flags.go:64] FLAG: --pods-per-core="0" Sep 08 08:47:32.568012 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559963 2554 flags.go:64] FLAG: --port="10250" Sep 08 08:47:32.568012 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559965 2554 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 08 08:47:32.568012 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559968 2554 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02dd1941e50aaa51d" Sep 08 08:47:32.568012 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559971 2554 flags.go:64] FLAG: --qos-reserved="" Sep 08 08:47:32.568012 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559976 2554 flags.go:64] FLAG: --read-only-port="10255" Sep 08 08:47:32.568012 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559978 2554 flags.go:64] FLAG: --register-node="true" Sep 08 08:47:32.568012 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559981 2554 flags.go:64] FLAG: --register-with-taints="" Sep 08 08:47:32.568012 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559985 2554 flags.go:64] FLAG: --registry-burst="10" Sep 08 08:47:32.568012 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559987 2554 flags.go:64] FLAG: --registry-qps="5" Sep 08 08:47:32.568012 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559990 2554 flags.go:64] FLAG: --reserved-cpus="" Sep 08 08:47:32.568012 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559993 2554 flags.go:64] FLAG: --reserved-memory="" Sep 08 08:47:32.568012 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559996 2554 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 08 08:47:32.568012 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.559999 2554 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 08 08:47:32.568012 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.560002 2554 flags.go:64] FLAG: --rotate-certificates="false" Sep 08 08:47:32.568012 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.560005 2554 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 08 08:47:32.568012 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.560008 2554 flags.go:64] FLAG: --runonce="false" Sep 08 08:47:32.568012 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.560010 2554 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 08 08:47:32.568708 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.560013 2554 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 08 08:47:32.568708 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.560016 2554 flags.go:64] FLAG: --seccomp-default="false" Sep 08 08:47:32.568708 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.560019 2554 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 08 08:47:32.568708 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.560022 2554 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 08 08:47:32.568708 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.560025 2554 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 08 08:47:32.568708 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.560027 2554 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 08 08:47:32.568708 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.560030 2554 flags.go:64] FLAG: --storage-driver-password="root" Sep 08 08:47:32.568708 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.560034 2554 flags.go:64] FLAG: --storage-driver-secure="false" Sep 08 08:47:32.568708 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.560037 2554 flags.go:64] FLAG: --storage-driver-table="stats" Sep 08 08:47:32.568708 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.560040 2554 flags.go:64] FLAG: --storage-driver-user="root" Sep 08 08:47:32.568708 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.560042 2554 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 08 08:47:32.568708 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.560045 2554 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 08 08:47:32.568708 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.560048 2554 flags.go:64] FLAG: --system-cgroups="" Sep 08 08:47:32.568708 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.560052 2554 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 08 08:47:32.568708 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.560057 2554 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 08 08:47:32.568708 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.560059 2554 flags.go:64] FLAG: --tls-cert-file="" Sep 08 08:47:32.568708 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.560062 2554 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 08 08:47:32.568708 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.560066 2554 flags.go:64] FLAG: --tls-min-version="" Sep 08 08:47:32.568708 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.560068 2554 flags.go:64] FLAG: --tls-private-key-file="" Sep 08 08:47:32.568708 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.560086 2554 flags.go:64] FLAG: --topology-manager-policy="none" Sep 08 08:47:32.568708 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.560089 2554 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 08 08:47:32.568708 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.560092 2554 flags.go:64] FLAG: --topology-manager-scope="container" Sep 08 08:47:32.568708 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.560095 2554 flags.go:64] FLAG: --v="2" Sep 08 08:47:32.568708 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.560099 2554 flags.go:64] FLAG: --version="false" Sep 08 08:47:32.568708 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.560102 2554 flags.go:64] FLAG: --vmodule="" Sep 08 08:47:32.569335 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.560106 2554 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 08 08:47:32.569335 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.560109 2554 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 08 08:47:32.569335 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560215 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 08:47:32.569335 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560219 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 08:47:32.569335 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560222 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 08:47:32.569335 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560226 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 08:47:32.569335 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560229 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 08:47:32.569335 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560231 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 08:47:32.569335 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560234 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 08:47:32.569335 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560237 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 08:47:32.569335 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560240 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 08:47:32.569335 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560242 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 08:47:32.569335 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560245 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 08:47:32.569335 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560248 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 08:47:32.569335 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560252 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 08:47:32.569335 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560254 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 08:47:32.569335 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560257 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 08:47:32.569335 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560260 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 08:47:32.569335 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560262 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 08:47:32.569897 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560265 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 08:47:32.569897 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560267 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 08:47:32.569897 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560271 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 08:47:32.569897 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560273 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 08:47:32.569897 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560276 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 08:47:32.569897 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560279 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 08:47:32.569897 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560283 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 08:47:32.569897 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560287 2554 feature_gate.go:328] unrecognized feature gate: Example Sep 08 08:47:32.569897 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560289 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 08:47:32.569897 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560292 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 08:47:32.569897 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560295 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 08:47:32.569897 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560297 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 08:47:32.569897 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560300 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 08:47:32.569897 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560302 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 08:47:32.569897 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560305 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 08:47:32.569897 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560308 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 08:47:32.569897 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560310 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 08:47:32.569897 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560313 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 08:47:32.569897 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560315 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 08:47:32.569897 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560318 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 08:47:32.570426 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560321 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 08:47:32.570426 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560324 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 08:47:32.570426 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560326 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 08:47:32.570426 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560329 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 08:47:32.570426 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560331 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 08:47:32.570426 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560334 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 08:47:32.570426 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560337 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 08:47:32.570426 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560340 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 08:47:32.570426 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560343 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 08:47:32.570426 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560346 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 08:47:32.570426 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560348 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 08:47:32.570426 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560351 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 08:47:32.570426 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560353 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 08:47:32.570426 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560356 2554 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 08:47:32.570426 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560360 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 08:47:32.570426 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560364 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 08:47:32.570426 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560367 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 08:47:32.570426 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560370 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 08:47:32.570426 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560373 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 08:47:32.570426 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560377 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 08:47:32.570943 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560381 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 08:47:32.570943 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560384 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 08:47:32.570943 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560387 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 08:47:32.570943 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560390 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 08:47:32.570943 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560392 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 08:47:32.570943 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560396 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 08:47:32.570943 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560398 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 08:47:32.570943 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560401 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 08:47:32.570943 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560404 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 08:47:32.570943 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560407 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 08:47:32.570943 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560410 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 08:47:32.570943 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560413 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 08:47:32.570943 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560415 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 08:47:32.570943 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560418 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 08:47:32.570943 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560420 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 08:47:32.570943 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560423 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 08:47:32.570943 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560425 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 08:47:32.570943 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560428 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 08:47:32.570943 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560430 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 08:47:32.571432 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560434 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 08:47:32.571432 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560437 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 08:47:32.571432 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560439 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 08:47:32.571432 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560442 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 08:47:32.571432 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560445 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 08:47:32.571432 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560447 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 08:47:32.571432 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560450 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 08:47:32.571432 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560452 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 08:47:32.571432 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560455 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 08:47:32.571432 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560458 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 08:47:32.571432 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560460 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 08:47:32.571432 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560463 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 08:47:32.571432 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560466 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 08:47:32.571432 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560469 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 08:47:32.571432 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560472 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 08:47:32.571432 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560474 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 08:47:32.571432 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560477 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 08:47:32.571432 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560480 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 08:47:32.571432 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560482 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 08:47:32.571432 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560485 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 08:47:32.571914 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560487 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 08:47:32.571914 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560490 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 08:47:32.571914 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.560492 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 08:47:32.571914 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.561348 2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 08 08:47:32.571914 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.571002 2554 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 08 08:47:32.571914 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.571144 2554 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 08 08:47:32.571914 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.571159 2554 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 08 08:47:32.571914 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.571164 2554 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 08 08:47:32.571914 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571219 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 08:47:32.571914 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571225 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 08:47:32.571914 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571229 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 08:47:32.571914 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571232 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 08:47:32.571914 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571234 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 08:47:32.571914 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571237 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 08:47:32.571914 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571240 2554 feature_gate.go:328] unrecognized feature gate: Example Sep 08 08:47:32.572375 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571243 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 08:47:32.572375 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571246 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 08:47:32.572375 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571248 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 08:47:32.572375 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571251 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 08:47:32.572375 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571253 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 08:47:32.572375 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571256 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 08:47:32.572375 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571259 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 08:47:32.572375 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571261 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 08:47:32.572375 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571264 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 08:47:32.572375 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571266 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 08:47:32.572375 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571269 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 08:47:32.572375 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571272 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 08:47:32.572375 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571276 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 08:47:32.572375 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571280 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 08:47:32.572375 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571283 2554 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 08:47:32.572375 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571287 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 08:47:32.572375 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571289 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 08:47:32.572375 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571292 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 08:47:32.572375 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571294 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 08:47:32.572375 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571297 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 08:47:32.572855 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571300 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 08:47:32.572855 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571302 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 08:47:32.572855 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571306 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 08:47:32.572855 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571310 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 08:47:32.572855 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571313 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 08:47:32.572855 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571316 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 08:47:32.572855 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571318 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 08:47:32.572855 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571321 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 08:47:32.572855 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571323 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 08:47:32.572855 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571326 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 08:47:32.572855 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571329 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 08:47:32.572855 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571331 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 08:47:32.572855 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571334 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 08:47:32.572855 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571336 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 08:47:32.572855 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571339 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 08:47:32.572855 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571341 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 08:47:32.572855 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571344 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 08:47:32.572855 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571346 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 08:47:32.572855 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571349 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 08:47:32.573323 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571352 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 08:47:32.573323 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571355 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 08:47:32.573323 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571358 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 08:47:32.573323 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571361 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 08:47:32.573323 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571363 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 08:47:32.573323 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571366 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 08:47:32.573323 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571368 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 08:47:32.573323 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571371 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 08:47:32.573323 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571374 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 08:47:32.573323 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571376 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 08:47:32.573323 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571379 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 08:47:32.573323 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571381 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 08:47:32.573323 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571384 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 08:47:32.573323 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571386 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 08:47:32.573323 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571389 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 08:47:32.573323 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571392 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 08:47:32.573323 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571395 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 08:47:32.573323 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571397 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 08:47:32.573323 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571400 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 08:47:32.573323 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571402 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 08:47:32.573799 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571405 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 08:47:32.573799 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571407 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 08:47:32.573799 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571410 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 08:47:32.573799 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571413 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 08:47:32.573799 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571415 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 08:47:32.573799 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571418 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 08:47:32.573799 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571420 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 08:47:32.573799 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571423 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 08:47:32.573799 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571425 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 08:47:32.573799 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571428 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 08:47:32.573799 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571430 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 08:47:32.573799 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571433 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 08:47:32.573799 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571436 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 08:47:32.573799 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571439 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 08:47:32.573799 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571442 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 08:47:32.573799 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571445 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 08:47:32.573799 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571448 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 08:47:32.573799 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571451 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 08:47:32.573799 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571453 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 08:47:32.573799 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571456 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 08:47:32.574376 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571458 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 08:47:32.574376 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571461 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 08:47:32.574376 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571463 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 08:47:32.574376 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571466 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 08:47:32.574376 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571468 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 08:47:32.574376 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571472 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 08:47:32.574376 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571474 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 08:47:32.574376 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571477 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 08:47:32.574376 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571479 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 08:47:32.574376 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571482 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 08:47:32.574376 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571486 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 08:47:32.574376 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571489 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 08:47:32.574376 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571492 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 08:47:32.574376 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.571497 2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 08 08:47:32.574717 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571601 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 08:47:32.574717 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571605 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 08:47:32.574717 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571609 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 08:47:32.574717 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571612 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 08:47:32.574717 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571614 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 08:47:32.574717 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571617 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 08:47:32.574717 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571620 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 08:47:32.574717 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571622 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 08:47:32.574717 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571625 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 08:47:32.574717 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571628 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 08:47:32.574717 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571630 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 08:47:32.574717 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571633 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 08:47:32.574717 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571636 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 08:47:32.574717 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571639 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 08:47:32.574717 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571642 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 08:47:32.574717 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571644 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 08:47:32.574717 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571647 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 08:47:32.574717 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571649 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 08:47:32.574717 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571652 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 08:47:32.574717 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571654 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 08:47:32.575313 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571657 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 08:47:32.575313 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571659 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 08:47:32.575313 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571662 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 08:47:32.575313 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571664 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 08:47:32.575313 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571667 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 08:47:32.575313 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571669 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 08:47:32.575313 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571671 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 08:47:32.575313 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571674 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 08:47:32.575313 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571676 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 08:47:32.575313 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571679 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 08:47:32.575313 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571681 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 08:47:32.575313 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571684 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 08:47:32.575313 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571686 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 08:47:32.575313 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571689 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 08:47:32.575313 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571693 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 08:47:32.575313 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571697 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 08:47:32.575313 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571699 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 08:47:32.575313 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571702 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 08:47:32.575313 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571704 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 08:47:32.575770 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571707 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 08:47:32.575770 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571709 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 08:47:32.575770 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571712 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 08:47:32.575770 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571715 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 08:47:32.575770 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571717 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 08:47:32.575770 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571719 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 08:47:32.575770 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571722 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 08:47:32.575770 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571725 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 08:47:32.575770 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571728 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 08:47:32.575770 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571730 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 08:47:32.575770 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571734 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 08:47:32.575770 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571737 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 08:47:32.575770 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571740 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 08:47:32.575770 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571742 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 08:47:32.575770 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571745 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 08:47:32.575770 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571750 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 08:47:32.575770 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571752 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 08:47:32.575770 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571755 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 08:47:32.575770 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571758 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 08:47:32.575770 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571760 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 08:47:32.576277 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571763 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 08:47:32.576277 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571765 2554 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 08:47:32.576277 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571768 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 08:47:32.576277 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571770 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 08:47:32.576277 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571773 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 08:47:32.576277 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571775 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 08:47:32.576277 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571778 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 08:47:32.576277 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571780 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 08:47:32.576277 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571783 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 08:47:32.576277 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571785 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 08:47:32.576277 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571788 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 08:47:32.576277 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571790 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 08:47:32.576277 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571793 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 08:47:32.576277 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571795 2554 feature_gate.go:328] unrecognized feature gate: Example Sep 08 08:47:32.576277 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571798 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 08:47:32.576277 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571800 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 08:47:32.576277 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571802 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 08:47:32.576277 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571805 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 08:47:32.576277 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571807 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 08:47:32.576277 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571812 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 08:47:32.576759 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571815 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 08:47:32.576759 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571817 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 08:47:32.576759 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571820 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 08:47:32.576759 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571823 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 08:47:32.576759 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571825 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 08:47:32.576759 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571828 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 08:47:32.576759 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571831 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 08:47:32.576759 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571835 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 08:47:32.576759 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571837 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 08:47:32.576759 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571840 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 08:47:32.576759 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571843 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 08:47:32.576759 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571845 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 08:47:32.576759 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571848 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 08:47:32.576759 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571852 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 08:47:32.576759 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571855 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 08:47:32.576759 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571858 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 08:47:32.576759 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571861 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 08:47:32.576759 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571864 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 08:47:32.576759 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571866 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 08:47:32.577345 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:32.571869 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 08:47:32.577345 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.571874 2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 08 08:47:32.577345 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.571968 2554 server.go:962] "Client rotation is on, will bootstrap in background" Sep 08 08:47:32.577345 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.574010 2554 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 08 08:47:32.577345 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.574933 2554 server.go:1019] "Starting client certificate rotation" Sep 08 08:47:32.577345 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.575040 2554 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 08 08:47:32.577345 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.575097 2554 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 08 08:47:32.578321 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.578305 2554 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 08 08:47:32.579502 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.579483 2554 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 08 08:47:32.590852 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.590834 2554 log.go:25] "Validated CRI v1 runtime API" Sep 08 08:47:32.596621 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.596600 2554 log.go:25] "Validated CRI v1 image API" Sep 08 08:47:32.597569 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.597553 2554 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 08 08:47:32.602404 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.602377 2554 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 98edfbbc-ec1c-459c-b244-d8fb94698a73:/dev/nvme0n1p4 c000acd4-5690-4d82-879b-44a87cd0db72:/dev/nvme0n1p3] Sep 08 08:47:32.602404 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.602399 2554 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 08 08:47:32.608663 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.608645 2554 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 08 08:47:32.609022 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.608903 2554 manager.go:216] Machine: {Timestamp:2026-09-08 08:47:32.607537315 +0000 UTC m=+0.341546477 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3086750 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec264a080ee1c56c8a91700aa6d34515 SystemUUID:ec264a08-0ee1-c56c-8a91-700aa6d34515 BootID:d208a7d5-bacc-44b3-84ff-e78d5777b532 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:42:2f:99:2d:bf Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:42:2f:99:2d:bf Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d2:b8:d0:49:65:64 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 08 08:47:32.609022 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.609009 2554 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 08 08:47:32.609177 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.609132 2554 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 08 08:47:32.612840 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.612818 2554 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 08 08:47:32.612970 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.612844 2554 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-177.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 08 08:47:32.614316 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.614305 2554 topology_manager.go:138] "Creating topology manager with none policy" Sep 08 08:47:32.614357 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.614318 2554 container_manager_linux.go:309] "Creating device plugin manager" Sep 08 08:47:32.614357 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.614326 2554 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 08 08:47:32.615133 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.615124 2554 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 08 08:47:32.616052 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.616043 2554 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 08 08:47:32.620028 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.620016 2554 state_mem.go:36] "Initialized new in-memory state store" Sep 08 08:47:32.620150 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.620140 2554 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 08 08:47:32.620182 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.620172 2554 kubelet.go:486] "Attempting to sync node with API server" Sep 08 08:47:32.620214 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.620185 2554 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 08 08:47:32.620214 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.620203 2554 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 08 08:47:32.620268 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.620234 2554 kubelet.go:398] "Adding apiserver pod source" Sep 08 08:47:32.620268 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.620249 2554 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 08 08:47:32.620651 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.620641 2554 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 08 08:47:32.620685 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.620659 2554 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 08 08:47:32.622851 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.622835 2554 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 08 08:47:32.625311 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.625299 2554 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 08 08:47:32.625353 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.625328 2554 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 08 08:47:32.627388 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.627375 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 08 08:47:32.627429 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.627394 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 08 08:47:32.627429 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.627400 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 08 08:47:32.627429 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.627405 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 08 08:47:32.627429 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.627415 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 08 08:47:32.627429 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.627423 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 08 08:47:32.627569 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.627440 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 08 08:47:32.627569 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.627446 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 08 08:47:32.627569 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.627452 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 08 08:47:32.627569 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.627458 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 08 08:47:32.627569 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.627467 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 08 08:47:32.627569 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.627472 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 08 08:47:32.628268 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.628257 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 08 08:47:32.628268 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.628268 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 08 08:47:32.632009 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.631997 2554 server.go:1268] "Started kubelet" Sep 08 08:47:32.632147 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:32.632127 2554 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-177.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 08 08:47:32.632215 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:32.632128 2554 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 08 08:47:32.632215 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.632161 2554 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 08 08:47:32.632215 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.632173 2554 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 08 08:47:32.632353 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.632237 2554 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 08 08:47:32.632353 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.632168 2554 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-177.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 08 08:47:32.632875 ip-10-0-141-177 systemd[1]: Started Kubernetes Kubelet. Sep 08 08:47:32.634289 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.634262 2554 server.go:310] "Adding debug handlers to kubelet server" Sep 08 08:47:32.635713 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.635486 2554 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 08 08:47:32.638657 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:32.638637 2554 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 08 08:47:32.640403 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.640387 2554 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 08 08:47:32.640403 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.640398 2554 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 08 08:47:32.640634 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:32.640615 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-177.ec2.internal\" not found" Sep 08 08:47:32.641120 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.641103 2554 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 08 08:47:32.641446 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.641308 2554 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 08 08:47:32.641514 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.641451 2554 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 08 08:47:32.641630 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.641618 2554 reconstruct.go:97] "Volume reconstruction finished" Sep 08 08:47:32.641630 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.641630 2554 reconciler.go:29] "Reconciler: start to sync state" Sep 08 08:47:32.641954 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.641936 2554 factory.go:55] Registering systemd factory Sep 08 08:47:32.642027 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.641956 2554 factory.go:223] Registration of the systemd container factory successfully Sep 08 08:47:32.642319 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.642288 2554 factory.go:153] Registering CRI-O factory Sep 08 08:47:32.642319 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.642307 2554 factory.go:223] Registration of the crio container factory successfully Sep 08 08:47:32.642521 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.642506 2554 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 08 08:47:32.642583 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.642531 2554 factory.go:103] Registering Raw factory Sep 08 08:47:32.642583 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.642549 2554 manager.go:1196] Started watching for new ooms in manager Sep 08 08:47:32.642993 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.642980 2554 manager.go:349] Starting recovery of all containers Sep 08 08:47:32.646125 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:32.641621 2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-177.ec2.internal.18d34c3189029af0 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-177.ec2.internal,UID:ip-10-0-141-177.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-177.ec2.internal,},FirstTimestamp:2026-09-08 08:47:32.631976688 +0000 UTC m=+0.365985843,LastTimestamp:2026-09-08 08:47:32.631976688 +0000 UTC m=+0.365985843,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-177.ec2.internal,}" Sep 08 08:47:32.648723 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:32.648687 2554 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-177.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 08 08:47:32.648823 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:32.648789 2554 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 08 08:47:32.656045 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.656029 2554 manager.go:354] Recovery completed Sep 08 08:47:32.658561 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:32.658537 2554 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Sep 08 08:47:32.660105 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.660087 2554 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-n5vq8" Sep 08 08:47:32.661903 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.661886 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 08:47:32.663613 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.663597 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-177.ec2.internal" event="NodeHasSufficientMemory" Sep 08 08:47:32.663698 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.663627 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-177.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 08:47:32.663698 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.663642 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-177.ec2.internal" event="NodeHasSufficientPID" Sep 08 08:47:32.664241 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.664225 2554 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 08 08:47:32.664241 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.664241 2554 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 08 08:47:32.664386 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.664261 2554 state_mem.go:36] "Initialized new in-memory state store" Sep 08 08:47:32.664386 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.664231 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 08:47:32.665023 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.665006 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-177.ec2.internal" event="NodeHasSufficientMemory" Sep 08 08:47:32.665115 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.665035 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-177.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 08:47:32.665115 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.665047 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-177.ec2.internal" event="NodeHasSufficientPID" Sep 08 08:47:32.665699 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:32.665634 2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-177.ec2.internal.18d34c318ae55301 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-177.ec2.internal,UID:ip-10-0-141-177.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-177.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-177.ec2.internal,},FirstTimestamp:2026-09-08 08:47:32.663612161 +0000 UTC m=+0.397621320,LastTimestamp:2026-09-08 08:47:32.663612161 +0000 UTC m=+0.397621320,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-177.ec2.internal,}" Sep 08 08:47:32.665962 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.665945 2554 policy_none.go:49] "None policy: Start" Sep 08 08:47:32.666023 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.665967 2554 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 08 08:47:32.666023 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.665979 2554 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 08 08:47:32.667184 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.667173 2554 policy_none.go:47] "Start" Sep 08 08:47:32.668123 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.668107 2554 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-n5vq8" Sep 08 08:47:32.710713 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.709722 2554 manager.go:337] "Starting Device Plugin manager" Sep 08 08:47:32.710713 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:32.709756 2554 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 08 08:47:32.710713 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.709766 2554 server.go:87] "Starting device plugin registration server" Sep 08 08:47:32.710713 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.709991 2554 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 08 08:47:32.710713 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.710003 2554 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 08 08:47:32.710713 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.710129 2554 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 08 08:47:32.710713 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.710288 2554 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 08 08:47:32.710713 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.710308 2554 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 08 08:47:32.711145 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:32.710750 2554 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 08 08:47:32.711145 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:32.710794 2554 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-177.ec2.internal\" not found" Sep 08 08:47:32.799113 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.799084 2554 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 08 08:47:32.800337 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.800320 2554 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 08 08:47:32.800458 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.800378 2554 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 08 08:47:32.800458 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.800402 2554 kubelet.go:2448] "Starting kubelet main sync loop" Sep 08 08:47:32.800458 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:32.800444 2554 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 08 08:47:32.803033 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.803015 2554 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 08:47:32.810629 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.810558 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 08:47:32.811383 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.811370 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-177.ec2.internal" event="NodeHasSufficientMemory" Sep 08 08:47:32.811438 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.811397 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-177.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 08:47:32.811438 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.811407 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-177.ec2.internal" event="NodeHasSufficientPID" Sep 08 08:47:32.811438 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.811426 2554 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-177.ec2.internal" Sep 08 08:47:32.828885 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.828867 2554 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-177.ec2.internal" Sep 08 08:47:32.828948 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:32.828888 2554 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-177.ec2.internal\": node \"ip-10-0-141-177.ec2.internal\" not found" Sep 08 08:47:32.872532 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:32.872512 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-177.ec2.internal\" not found" Sep 08 08:47:32.901497 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.901470 2554 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-177.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-177.ec2.internal"] Sep 08 08:47:32.901554 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.901545 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 08:47:32.902837 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.902823 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-177.ec2.internal" event="NodeHasSufficientMemory" Sep 08 08:47:32.902884 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.902856 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-177.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 08:47:32.902884 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.902866 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-177.ec2.internal" event="NodeHasSufficientPID" Sep 08 08:47:32.904172 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.904160 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 08:47:32.904285 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.904272 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-177.ec2.internal" Sep 08 08:47:32.904334 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.904298 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 08:47:32.904811 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.904796 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-177.ec2.internal" event="NodeHasSufficientMemory" Sep 08 08:47:32.904875 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.904828 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-177.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 08:47:32.904875 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.904839 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-177.ec2.internal" event="NodeHasSufficientPID" Sep 08 08:47:32.904875 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.904858 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-177.ec2.internal" event="NodeHasSufficientMemory" Sep 08 08:47:32.905006 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.904882 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-177.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 08:47:32.905006 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.904897 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-177.ec2.internal" event="NodeHasSufficientPID" Sep 08 08:47:32.906294 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.906272 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-177.ec2.internal" Sep 08 08:47:32.906373 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.906305 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 08:47:32.906993 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.906969 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-177.ec2.internal" event="NodeHasSufficientMemory" Sep 08 08:47:32.907066 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.906996 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-177.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 08:47:32.907066 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.907009 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-177.ec2.internal" event="NodeHasSufficientPID" Sep 08 08:47:32.928653 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:32.928633 2554 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-177.ec2.internal\" not found" node="ip-10-0-141-177.ec2.internal" Sep 08 08:47:32.932101 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:32.932087 2554 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-177.ec2.internal\" not found" node="ip-10-0-141-177.ec2.internal" Sep 08 08:47:32.943862 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.943844 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/96d81d290f44eaca1c0dbdec93010d30-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-177.ec2.internal\" (UID: \"96d81d290f44eaca1c0dbdec93010d30\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-177.ec2.internal" Sep 08 08:47:32.943959 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.943869 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96d81d290f44eaca1c0dbdec93010d30-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-177.ec2.internal\" (UID: \"96d81d290f44eaca1c0dbdec93010d30\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-177.ec2.internal" Sep 08 08:47:32.943959 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:32.943886 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fdf29c3cd2e403aedd9a5d1cadaf161a-config\") pod \"kube-apiserver-proxy-ip-10-0-141-177.ec2.internal\" (UID: \"fdf29c3cd2e403aedd9a5d1cadaf161a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-177.ec2.internal" Sep 08 08:47:32.973600 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:32.973583 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-177.ec2.internal\" not found" Sep 08 08:47:33.044403 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:33.044381 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fdf29c3cd2e403aedd9a5d1cadaf161a-config\") pod \"kube-apiserver-proxy-ip-10-0-141-177.ec2.internal\" (UID: \"fdf29c3cd2e403aedd9a5d1cadaf161a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-177.ec2.internal" Sep 08 08:47:33.044485 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:33.044408 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/96d81d290f44eaca1c0dbdec93010d30-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-177.ec2.internal\" (UID: \"96d81d290f44eaca1c0dbdec93010d30\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-177.ec2.internal" Sep 08 08:47:33.044485 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:33.044459 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96d81d290f44eaca1c0dbdec93010d30-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-177.ec2.internal\" (UID: \"96d81d290f44eaca1c0dbdec93010d30\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-177.ec2.internal" Sep 08 08:47:33.044546 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:33.044509 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96d81d290f44eaca1c0dbdec93010d30-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-177.ec2.internal\" (UID: \"96d81d290f44eaca1c0dbdec93010d30\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-177.ec2.internal" Sep 08 08:47:33.044581 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:33.044544 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fdf29c3cd2e403aedd9a5d1cadaf161a-config\") pod \"kube-apiserver-proxy-ip-10-0-141-177.ec2.internal\" (UID: \"fdf29c3cd2e403aedd9a5d1cadaf161a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-177.ec2.internal" Sep 08 08:47:33.044613 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:33.044580 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/96d81d290f44eaca1c0dbdec93010d30-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-177.ec2.internal\" (UID: \"96d81d290f44eaca1c0dbdec93010d30\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-177.ec2.internal" Sep 08 08:47:33.074522 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:33.074470 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-177.ec2.internal\" not found" Sep 08 08:47:33.175245 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:33.175225 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-177.ec2.internal\" not found" Sep 08 08:47:33.231402 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:33.231386 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-177.ec2.internal" Sep 08 08:47:33.236310 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:33.235689 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-177.ec2.internal" Sep 08 08:47:33.275838 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:33.275818 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-177.ec2.internal\" not found" Sep 08 08:47:34.602232 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:34.602213 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-177.ec2.internal\" not found" Sep 08 08:47:34.702665 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:34.702642 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-177.ec2.internal\" not found" Sep 08 08:47:34.800213 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:34.800187 2554 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 08 08:47:34.800569 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:34.800322 2554 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 08 08:47:34.803319 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:34.803302 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-177.ec2.internal\" not found" Sep 08 08:47:34.874928 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:34.874876 2554 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 08:47:34.898615 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:34.898590 2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-07 08:42:33 +0000 UTC" deadline="2028-06-05 05:43:23.020582486 +0000 UTC" Sep 08 08:47:34.898615 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:34.898612 2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15260h55m48.121972824s" Sep 08 08:47:34.966905 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:34.966890 2554 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-177.ec2.internal" Sep 08 08:47:34.977191 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:34.977173 2554 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 08 08:47:34.979594 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:34.979583 2554 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-177.ec2.internal" Sep 08 08:47:34.991024 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:34.991011 2554 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 08 08:47:35.022256 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.022240 2554 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 08:47:35.271707 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:35.271550 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96d81d290f44eaca1c0dbdec93010d30.slice/crio-c9f2673d2c833acded65d28efc40b1e8edc385bc76023df777e27050b6197424 WatchSource:0}: Error finding container c9f2673d2c833acded65d28efc40b1e8edc385bc76023df777e27050b6197424: Status 404 returned error can't find the container with id c9f2673d2c833acded65d28efc40b1e8edc385bc76023df777e27050b6197424 Sep 08 08:47:35.277005 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.276989 2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 08 08:47:35.297916 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:35.297896 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdf29c3cd2e403aedd9a5d1cadaf161a.slice/crio-b0fa81a81075bb2e967002e9da65ac61e87089af8f28a9ed3cd3a6ad07a3162e WatchSource:0}: Error finding container b0fa81a81075bb2e967002e9da65ac61e87089af8f28a9ed3cd3a6ad07a3162e: Status 404 returned error can't find the container with id b0fa81a81075bb2e967002e9da65ac61e87089af8f28a9ed3cd3a6ad07a3162e Sep 08 08:47:35.426992 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.426930 2554 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 08:47:35.619928 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.619904 2554 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 08:47:35.847220 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.847195 2554 apiserver.go:52] "Watching apiserver" Sep 08 08:47:35.855104 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.855081 2554 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 08 08:47:35.855841 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.855530 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-8bx7f","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-177.ec2.internal","openshift-multus/multus-additional-cni-plugins-wl97n","openshift-multus/multus-txfv4","openshift-network-diagnostics/network-check-target-z4smd","kube-system/global-pull-secret-syncer-b6s4m","kube-system/kube-apiserver-proxy-ip-10-0-141-177.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkwv8","openshift-multus/network-metrics-daemon-9xk7m","openshift-network-operator/iptables-alerter-nsz5v","openshift-ovn-kubernetes/ovnkube-node-xn2lh","kube-system/konnectivity-agent-lmwtf","openshift-cluster-node-tuning-operator/tuned-gxjvt","openshift-dns/node-resolver-lbvgv"] Sep 08 08:47:35.857271 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.857077 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8bx7f" Sep 08 08:47:35.858483 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.858461 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wl97n" Sep 08 08:47:35.859850 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.859830 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.860437 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.860419 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-mnrfd\"" Sep 08 08:47:35.860808 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.860788 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-24gc2\"" Sep 08 08:47:35.861183 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.861030 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 08 08:47:35.862098 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.861301 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 08 08:47:35.862098 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.861379 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4smd" Sep 08 08:47:35.862098 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.861474 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 08 08:47:35.862098 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:35.861468 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4smd" podUID="5e400e6f-9bee-43f4-b6b1-f75ba65c39fc" Sep 08 08:47:35.862098 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.861592 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 08 08:47:35.862098 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.861711 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 08 08:47:35.862098 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.861717 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 08 08:47:35.862098 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.861870 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 08 08:47:35.862098 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.861965 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 08 08:47:35.862098 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.862093 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 08 08:47:35.862628 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.862474 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rswsg\"" Sep 08 08:47:35.863374 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.863226 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6s4m" Sep 08 08:47:35.863374 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:35.863293 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6s4m" podUID="1746be3a-25ff-4dc5-b4fb-6075f46391e0" Sep 08 08:47:35.865258 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.865226 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkwv8" Sep 08 08:47:35.867418 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.867401 2554 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 08 08:47:35.868239 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.867950 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xk7m" Sep 08 08:47:35.868239 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:35.868112 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xk7m" podUID="57716ea8-fa5c-47c2-a186-7a0acbf6ae38" Sep 08 08:47:35.868239 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.867957 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 08 08:47:35.868478 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.868430 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-6wx7w\"" Sep 08 08:47:35.868935 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.868595 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 08 08:47:35.868935 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.868699 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 08 08:47:35.870102 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.870085 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nsz5v" Sep 08 08:47:35.873244 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.873205 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:35.873359 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.873339 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 08 08:47:35.873423 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.873380 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9nptz\"" Sep 08 08:47:35.874419 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.874402 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 08 08:47:35.874766 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.874665 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 08 08:47:35.875306 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.874929 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lmwtf" Sep 08 08:47:35.876924 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.876576 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:35.878550 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.878193 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 08 08:47:35.878550 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.878384 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-wzlsx\"" Sep 08 08:47:35.880033 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.878867 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lbvgv" Sep 08 08:47:35.880033 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.878945 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4e49d642-1176-4297-84fd-d0cef9884bff-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wl97n\" (UID: \"4e49d642-1176-4297-84fd-d0cef9884bff\") " pod="openshift-multus/multus-additional-cni-plugins-wl97n" Sep 08 08:47:35.880033 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.878992 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-host-run-netns\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.880033 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.879019 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-host-var-lib-cni-bin\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.880033 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.879042 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/095842a7-07af-47c9-a386-6a90456cc5c0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wkwv8\" (UID: \"095842a7-07af-47c9-a386-6a90456cc5c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkwv8" Sep 08 08:47:35.880033 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.879065 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/095842a7-07af-47c9-a386-6a90456cc5c0-registration-dir\") pod \"aws-ebs-csi-driver-node-wkwv8\" (UID: \"095842a7-07af-47c9-a386-6a90456cc5c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkwv8" Sep 08 08:47:35.880033 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.879087 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/095842a7-07af-47c9-a386-6a90456cc5c0-etc-selinux\") pod \"aws-ebs-csi-driver-node-wkwv8\" (UID: \"095842a7-07af-47c9-a386-6a90456cc5c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkwv8" Sep 08 08:47:35.880033 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.879110 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmvx2\" (UniqueName: \"kubernetes.io/projected/57716ea8-fa5c-47c2-a186-7a0acbf6ae38-kube-api-access-mmvx2\") pod \"network-metrics-daemon-9xk7m\" (UID: \"57716ea8-fa5c-47c2-a186-7a0acbf6ae38\") " pod="openshift-multus/network-metrics-daemon-9xk7m" Sep 08 08:47:35.880033 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.879136 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4e49d642-1176-4297-84fd-d0cef9884bff-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wl97n\" (UID: \"4e49d642-1176-4297-84fd-d0cef9884bff\") " pod="openshift-multus/multus-additional-cni-plugins-wl97n" Sep 08 08:47:35.880033 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.879164 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-etc-kubernetes\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.880033 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.879186 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4e49d642-1176-4297-84fd-d0cef9884bff-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wl97n\" (UID: \"4e49d642-1176-4297-84fd-d0cef9884bff\") " pod="openshift-multus/multus-additional-cni-plugins-wl97n" Sep 08 08:47:35.880033 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.879209 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bj2cn\" (UniqueName: \"kubernetes.io/projected/095842a7-07af-47c9-a386-6a90456cc5c0-kube-api-access-bj2cn\") pod \"aws-ebs-csi-driver-node-wkwv8\" (UID: \"095842a7-07af-47c9-a386-6a90456cc5c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkwv8" Sep 08 08:47:35.880033 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.879231 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57716ea8-fa5c-47c2-a186-7a0acbf6ae38-metrics-certs\") pod \"network-metrics-daemon-9xk7m\" (UID: \"57716ea8-fa5c-47c2-a186-7a0acbf6ae38\") " pod="openshift-multus/network-metrics-daemon-9xk7m" Sep 08 08:47:35.880033 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.879267 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/01944a8d-4aa3-4e4b-9df4-922d4b99f848-serviceca\") pod \"node-ca-8bx7f\" (UID: \"01944a8d-4aa3-4e4b-9df4-922d4b99f848\") " pod="openshift-image-registry/node-ca-8bx7f" Sep 08 08:47:35.880033 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.879289 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzzsb\" (UniqueName: \"kubernetes.io/projected/01944a8d-4aa3-4e4b-9df4-922d4b99f848-kube-api-access-jzzsb\") pod \"node-ca-8bx7f\" (UID: \"01944a8d-4aa3-4e4b-9df4-922d4b99f848\") " pod="openshift-image-registry/node-ca-8bx7f" Sep 08 08:47:35.880033 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.879319 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dq6cj\" (UniqueName: \"kubernetes.io/projected/5e400e6f-9bee-43f4-b6b1-f75ba65c39fc-kube-api-access-dq6cj\") pod \"network-check-target-z4smd\" (UID: \"5e400e6f-9bee-43f4-b6b1-f75ba65c39fc\") " pod="openshift-network-diagnostics/network-check-target-z4smd" Sep 08 08:47:35.880839 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.879345 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-host-var-lib-cni-multus\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.880839 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.879369 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4e49d642-1176-4297-84fd-d0cef9884bff-system-cni-dir\") pod \"multus-additional-cni-plugins-wl97n\" (UID: \"4e49d642-1176-4297-84fd-d0cef9884bff\") " pod="openshift-multus/multus-additional-cni-plugins-wl97n" Sep 08 08:47:35.880839 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.879391 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-host-run-k8s-cni-cncf-io\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.880839 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.879413 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-system-cni-dir\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.880839 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.879435 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-multus-cni-dir\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.880839 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.879458 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1746be3a-25ff-4dc5-b4fb-6075f46391e0-original-pull-secret\") pod \"global-pull-secret-syncer-b6s4m\" (UID: \"1746be3a-25ff-4dc5-b4fb-6075f46391e0\") " pod="kube-system/global-pull-secret-syncer-b6s4m" Sep 08 08:47:35.880839 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.879481 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/095842a7-07af-47c9-a386-6a90456cc5c0-device-dir\") pod \"aws-ebs-csi-driver-node-wkwv8\" (UID: \"095842a7-07af-47c9-a386-6a90456cc5c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkwv8" Sep 08 08:47:35.880839 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.879503 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/095842a7-07af-47c9-a386-6a90456cc5c0-sys-fs\") pod \"aws-ebs-csi-driver-node-wkwv8\" (UID: \"095842a7-07af-47c9-a386-6a90456cc5c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkwv8" Sep 08 08:47:35.880839 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.879524 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/01944a8d-4aa3-4e4b-9df4-922d4b99f848-host\") pod \"node-ca-8bx7f\" (UID: \"01944a8d-4aa3-4e4b-9df4-922d4b99f848\") " pod="openshift-image-registry/node-ca-8bx7f" Sep 08 08:47:35.880839 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.879545 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-host-run-multus-certs\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.880839 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.879566 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1746be3a-25ff-4dc5-b4fb-6075f46391e0-kubelet-config\") pod \"global-pull-secret-syncer-b6s4m\" (UID: \"1746be3a-25ff-4dc5-b4fb-6075f46391e0\") " pod="kube-system/global-pull-secret-syncer-b6s4m" Sep 08 08:47:35.880839 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.879586 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-cnibin\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.880839 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.879616 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-cni-binary-copy\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.880839 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.879640 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4e49d642-1176-4297-84fd-d0cef9884bff-cnibin\") pod \"multus-additional-cni-plugins-wl97n\" (UID: \"4e49d642-1176-4297-84fd-d0cef9884bff\") " pod="openshift-multus/multus-additional-cni-plugins-wl97n" Sep 08 08:47:35.880839 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.879670 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ld4l\" (UniqueName: \"kubernetes.io/projected/4e49d642-1176-4297-84fd-d0cef9884bff-kube-api-access-8ld4l\") pod \"multus-additional-cni-plugins-wl97n\" (UID: \"4e49d642-1176-4297-84fd-d0cef9884bff\") " pod="openshift-multus/multus-additional-cni-plugins-wl97n" Sep 08 08:47:35.880839 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.879699 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-multus-socket-dir-parent\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.881663 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.879725 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-host-var-lib-kubelet\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.881663 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.879754 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-hostroot\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.881663 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.879775 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-multus-conf-dir\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.881663 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.879799 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn5zp\" (UniqueName: \"kubernetes.io/projected/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-kube-api-access-cn5zp\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.881663 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.879819 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1746be3a-25ff-4dc5-b4fb-6075f46391e0-dbus\") pod \"global-pull-secret-syncer-b6s4m\" (UID: \"1746be3a-25ff-4dc5-b4fb-6075f46391e0\") " pod="kube-system/global-pull-secret-syncer-b6s4m" Sep 08 08:47:35.881663 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.879841 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-multus-daemon-config\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.881663 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.879863 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/095842a7-07af-47c9-a386-6a90456cc5c0-socket-dir\") pod \"aws-ebs-csi-driver-node-wkwv8\" (UID: \"095842a7-07af-47c9-a386-6a90456cc5c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkwv8" Sep 08 08:47:35.881663 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.879885 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4e49d642-1176-4297-84fd-d0cef9884bff-cni-binary-copy\") pod \"multus-additional-cni-plugins-wl97n\" (UID: \"4e49d642-1176-4297-84fd-d0cef9884bff\") " pod="openshift-multus/multus-additional-cni-plugins-wl97n" Sep 08 08:47:35.882069 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.881904 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 08 08:47:35.884534 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.882195 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 08 08:47:35.884534 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.882403 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 08 08:47:35.884534 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.882427 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 08 08:47:35.884534 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.882692 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-kcgjw\"" Sep 08 08:47:35.884534 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.882887 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 08 08:47:35.884534 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.883661 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 08 08:47:35.884534 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.883808 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-tmm6g\"" Sep 08 08:47:35.884534 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.883882 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 08 08:47:35.884534 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.883991 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 08 08:47:35.884534 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.884067 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 08 08:47:35.884534 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.884153 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 08 08:47:35.884534 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.884237 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-lcxlz\"" Sep 08 08:47:35.884534 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.884308 2554 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 08 08:47:35.884534 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.884462 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 08 08:47:35.901624 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.901604 2554 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-wsjm9" Sep 08 08:47:35.909150 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.909129 2554 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-wsjm9" Sep 08 08:47:35.967813 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.967781 2554 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 08 08:47:35.980954 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.980928 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bj2cn\" (UniqueName: \"kubernetes.io/projected/095842a7-07af-47c9-a386-6a90456cc5c0-kube-api-access-bj2cn\") pod \"aws-ebs-csi-driver-node-wkwv8\" (UID: \"095842a7-07af-47c9-a386-6a90456cc5c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkwv8" Sep 08 08:47:35.981068 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.980967 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57716ea8-fa5c-47c2-a186-7a0acbf6ae38-metrics-certs\") pod \"network-metrics-daemon-9xk7m\" (UID: \"57716ea8-fa5c-47c2-a186-7a0acbf6ae38\") " pod="openshift-multus/network-metrics-daemon-9xk7m" Sep 08 08:47:35.981068 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.981011 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dq6cj\" (UniqueName: \"kubernetes.io/projected/5e400e6f-9bee-43f4-b6b1-f75ba65c39fc-kube-api-access-dq6cj\") pod \"network-check-target-z4smd\" (UID: \"5e400e6f-9bee-43f4-b6b1-f75ba65c39fc\") " pod="openshift-network-diagnostics/network-check-target-z4smd" Sep 08 08:47:35.981068 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.981037 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f7affce-54c6-4ec7-8380-6cae8e0edd91-var-lib-kubelet\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:35.981068 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.981060 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-host-run-netns\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:35.981263 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.981114 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-host-var-lib-cni-multus\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.981263 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:35.981153 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 08:47:35.981263 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.981196 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9f7affce-54c6-4ec7-8380-6cae8e0edd91-run\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:35.981263 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:35.981227 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57716ea8-fa5c-47c2-a186-7a0acbf6ae38-metrics-certs podName:57716ea8-fa5c-47c2-a186-7a0acbf6ae38 nodeName:}" failed. No retries permitted until 2026-09-08 08:47:36.481206051 +0000 UTC m=+2.989312807 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57716ea8-fa5c-47c2-a186-7a0acbf6ae38-metrics-certs") pod "network-metrics-daemon-9xk7m" (UID: "57716ea8-fa5c-47c2-a186-7a0acbf6ae38") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 08:47:35.981263 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.981254 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ab4bc45b-8015-494b-b088-5f699f96acaa-host-slash\") pod \"iptables-alerter-nsz5v\" (UID: \"ab4bc45b-8015-494b-b088-5f699f96acaa\") " pod="openshift-network-operator/iptables-alerter-nsz5v" Sep 08 08:47:35.981504 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.981266 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-host-var-lib-cni-multus\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.981504 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.981319 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-multus-cni-dir\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.981504 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.981461 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/095842a7-07af-47c9-a386-6a90456cc5c0-sys-fs\") pod \"aws-ebs-csi-driver-node-wkwv8\" (UID: \"095842a7-07af-47c9-a386-6a90456cc5c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkwv8" Sep 08 08:47:35.981504 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.981493 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-host-slash\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:35.981689 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.981517 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-run-openvswitch\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:35.981689 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.981543 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/72a88ec1-2663-4148-9b6f-0b540b3fb77c-konnectivity-ca\") pod \"konnectivity-agent-lmwtf\" (UID: \"72a88ec1-2663-4148-9b6f-0b540b3fb77c\") " pod="kube-system/konnectivity-agent-lmwtf" Sep 08 08:47:35.981689 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.981566 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4db69aa-e494-477b-8263-f4c7d54f0d71-tmp-dir\") pod \"node-resolver-lbvgv\" (UID: \"a4db69aa-e494-477b-8263-f4c7d54f0d71\") " pod="openshift-dns/node-resolver-lbvgv" Sep 08 08:47:35.981689 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.981592 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-multus-daemon-config\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.981689 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.981615 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/095842a7-07af-47c9-a386-6a90456cc5c0-socket-dir\") pod \"aws-ebs-csi-driver-node-wkwv8\" (UID: \"095842a7-07af-47c9-a386-6a90456cc5c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkwv8" Sep 08 08:47:35.981895 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.981639 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ab4bc45b-8015-494b-b088-5f699f96acaa-iptables-alerter-script\") pod \"iptables-alerter-nsz5v\" (UID: \"ab4bc45b-8015-494b-b088-5f699f96acaa\") " pod="openshift-network-operator/iptables-alerter-nsz5v" Sep 08 08:47:35.981947 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.981909 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f7affce-54c6-4ec7-8380-6cae8e0edd91-etc-kubernetes\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:35.982021 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.981946 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9f7affce-54c6-4ec7-8380-6cae8e0edd91-host\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:35.982021 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.981999 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1746be3a-25ff-4dc5-b4fb-6075f46391e0-kubelet-config\") pod \"global-pull-secret-syncer-b6s4m\" (UID: \"1746be3a-25ff-4dc5-b4fb-6075f46391e0\") " pod="kube-system/global-pull-secret-syncer-b6s4m" Sep 08 08:47:35.982156 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.982040 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-cni-binary-copy\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.984278 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.982076 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-multus-conf-dir\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.984278 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.982266 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmvx2\" (UniqueName: \"kubernetes.io/projected/57716ea8-fa5c-47c2-a186-7a0acbf6ae38-kube-api-access-mmvx2\") pod \"network-metrics-daemon-9xk7m\" (UID: \"57716ea8-fa5c-47c2-a186-7a0acbf6ae38\") " pod="openshift-multus/network-metrics-daemon-9xk7m" Sep 08 08:47:35.984278 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.982301 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9f7affce-54c6-4ec7-8380-6cae8e0edd91-etc-modprobe-d\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:35.984278 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.982331 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4e49d642-1176-4297-84fd-d0cef9884bff-cnibin\") pod \"multus-additional-cni-plugins-wl97n\" (UID: \"4e49d642-1176-4297-84fd-d0cef9884bff\") " pod="openshift-multus/multus-additional-cni-plugins-wl97n" Sep 08 08:47:35.984278 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.982407 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-multus-cni-dir\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.984278 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.982359 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ld4l\" (UniqueName: \"kubernetes.io/projected/4e49d642-1176-4297-84fd-d0cef9884bff-kube-api-access-8ld4l\") pod \"multus-additional-cni-plugins-wl97n\" (UID: \"4e49d642-1176-4297-84fd-d0cef9884bff\") " pod="openshift-multus/multus-additional-cni-plugins-wl97n" Sep 08 08:47:35.984278 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.982507 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-node-log\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:35.984278 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.982509 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/095842a7-07af-47c9-a386-6a90456cc5c0-sys-fs\") pod \"aws-ebs-csi-driver-node-wkwv8\" (UID: \"095842a7-07af-47c9-a386-6a90456cc5c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkwv8" Sep 08 08:47:35.984278 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.982632 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1746be3a-25ff-4dc5-b4fb-6075f46391e0-kubelet-config\") pod \"global-pull-secret-syncer-b6s4m\" (UID: \"1746be3a-25ff-4dc5-b4fb-6075f46391e0\") " pod="kube-system/global-pull-secret-syncer-b6s4m" Sep 08 08:47:35.984278 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.982668 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-multus-conf-dir\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.984278 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.982740 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/095842a7-07af-47c9-a386-6a90456cc5c0-socket-dir\") pod \"aws-ebs-csi-driver-node-wkwv8\" (UID: \"095842a7-07af-47c9-a386-6a90456cc5c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkwv8" Sep 08 08:47:35.984278 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.983274 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-cni-binary-copy\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.984278 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.983728 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4e49d642-1176-4297-84fd-d0cef9884bff-cnibin\") pod \"multus-additional-cni-plugins-wl97n\" (UID: \"4e49d642-1176-4297-84fd-d0cef9884bff\") " pod="openshift-multus/multus-additional-cni-plugins-wl97n" Sep 08 08:47:35.984278 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.983793 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-host-cni-netd\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:35.984278 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.983834 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:35.984278 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.983891 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4e49d642-1176-4297-84fd-d0cef9884bff-cni-binary-copy\") pod \"multus-additional-cni-plugins-wl97n\" (UID: \"4e49d642-1176-4297-84fd-d0cef9884bff\") " pod="openshift-multus/multus-additional-cni-plugins-wl97n" Sep 08 08:47:35.984278 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.983939 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-host-run-netns\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.985093 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.983973 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/095842a7-07af-47c9-a386-6a90456cc5c0-etc-selinux\") pod \"aws-ebs-csi-driver-node-wkwv8\" (UID: \"095842a7-07af-47c9-a386-6a90456cc5c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkwv8" Sep 08 08:47:35.985093 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.984018 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-run-ovn\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:35.985093 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.984052 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-etc-kubernetes\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.985093 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.984084 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-ovnkube-script-lib\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:35.985093 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.984118 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzzsb\" (UniqueName: \"kubernetes.io/projected/01944a8d-4aa3-4e4b-9df4-922d4b99f848-kube-api-access-jzzsb\") pod \"node-ca-8bx7f\" (UID: \"01944a8d-4aa3-4e4b-9df4-922d4b99f848\") " pod="openshift-image-registry/node-ca-8bx7f" Sep 08 08:47:35.985093 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.984151 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9f7affce-54c6-4ec7-8380-6cae8e0edd91-etc-sysconfig\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:35.985093 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.984205 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9f7affce-54c6-4ec7-8380-6cae8e0edd91-etc-systemd\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:35.985093 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.984211 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/095842a7-07af-47c9-a386-6a90456cc5c0-etc-selinux\") pod \"aws-ebs-csi-driver-node-wkwv8\" (UID: \"095842a7-07af-47c9-a386-6a90456cc5c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkwv8" Sep 08 08:47:35.985093 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.984257 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-etc-kubernetes\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.985093 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.984355 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-host-kubelet\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:35.985093 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.984383 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-run-systemd\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:35.985093 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.984414 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-host-run-ovn-kubernetes\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:35.985093 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.984444 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwqjn\" (UniqueName: \"kubernetes.io/projected/a4db69aa-e494-477b-8263-f4c7d54f0d71-kube-api-access-wwqjn\") pod \"node-resolver-lbvgv\" (UID: \"a4db69aa-e494-477b-8263-f4c7d54f0d71\") " pod="openshift-dns/node-resolver-lbvgv" Sep 08 08:47:35.985093 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.984520 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9f7affce-54c6-4ec7-8380-6cae8e0edd91-sys\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:35.985093 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.984552 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9f7affce-54c6-4ec7-8380-6cae8e0edd91-lib-modules\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:35.985093 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.984611 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9f7affce-54c6-4ec7-8380-6cae8e0edd91-etc-tuned\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:35.985093 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.984644 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nq5qn\" (UniqueName: \"kubernetes.io/projected/9f7affce-54c6-4ec7-8380-6cae8e0edd91-kube-api-access-nq5qn\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:35.985858 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.984674 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-systemd-units\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:35.985858 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.984703 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4e49d642-1176-4297-84fd-d0cef9884bff-system-cni-dir\") pod \"multus-additional-cni-plugins-wl97n\" (UID: \"4e49d642-1176-4297-84fd-d0cef9884bff\") " pod="openshift-multus/multus-additional-cni-plugins-wl97n" Sep 08 08:47:35.985858 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.984774 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-host-run-netns\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.985858 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.984815 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-host-run-k8s-cni-cncf-io\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.985858 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.984849 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-log-socket\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:35.985858 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.984881 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a4db69aa-e494-477b-8263-f4c7d54f0d71-hosts-file\") pod \"node-resolver-lbvgv\" (UID: \"a4db69aa-e494-477b-8263-f4c7d54f0d71\") " pod="openshift-dns/node-resolver-lbvgv" Sep 08 08:47:35.985858 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.984914 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-system-cni-dir\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.985858 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.984919 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4e49d642-1176-4297-84fd-d0cef9884bff-cni-binary-copy\") pod \"multus-additional-cni-plugins-wl97n\" (UID: \"4e49d642-1176-4297-84fd-d0cef9884bff\") " pod="openshift-multus/multus-additional-cni-plugins-wl97n" Sep 08 08:47:35.985858 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.984943 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1746be3a-25ff-4dc5-b4fb-6075f46391e0-original-pull-secret\") pod \"global-pull-secret-syncer-b6s4m\" (UID: \"1746be3a-25ff-4dc5-b4fb-6075f46391e0\") " pod="kube-system/global-pull-secret-syncer-b6s4m" Sep 08 08:47:35.985858 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.984972 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4e49d642-1176-4297-84fd-d0cef9884bff-system-cni-dir\") pod \"multus-additional-cni-plugins-wl97n\" (UID: \"4e49d642-1176-4297-84fd-d0cef9884bff\") " pod="openshift-multus/multus-additional-cni-plugins-wl97n" Sep 08 08:47:35.985858 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.985033 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/095842a7-07af-47c9-a386-6a90456cc5c0-device-dir\") pod \"aws-ebs-csi-driver-node-wkwv8\" (UID: \"095842a7-07af-47c9-a386-6a90456cc5c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkwv8" Sep 08 08:47:35.985858 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.985055 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/01944a8d-4aa3-4e4b-9df4-922d4b99f848-host\") pod \"node-ca-8bx7f\" (UID: \"01944a8d-4aa3-4e4b-9df4-922d4b99f848\") " pod="openshift-image-registry/node-ca-8bx7f" Sep 08 08:47:35.985858 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.985081 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2rqz\" (UniqueName: \"kubernetes.io/projected/ab4bc45b-8015-494b-b088-5f699f96acaa-kube-api-access-t2rqz\") pod \"iptables-alerter-nsz5v\" (UID: \"ab4bc45b-8015-494b-b088-5f699f96acaa\") " pod="openshift-network-operator/iptables-alerter-nsz5v" Sep 08 08:47:35.985858 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.985099 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f7affce-54c6-4ec7-8380-6cae8e0edd91-tmp\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:35.985858 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.985107 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-system-cni-dir\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.985858 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.985117 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-var-lib-openvswitch\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:35.985858 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.985160 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/01944a8d-4aa3-4e4b-9df4-922d4b99f848-host\") pod \"node-ca-8bx7f\" (UID: \"01944a8d-4aa3-4e4b-9df4-922d4b99f848\") " pod="openshift-image-registry/node-ca-8bx7f" Sep 08 08:47:35.986635 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.985181 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-host-run-multus-certs\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.986635 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:35.985199 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 08:47:35.986635 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.985263 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-host-run-k8s-cni-cncf-io\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.986635 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.985302 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-cnibin\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.986635 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.985364 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/095842a7-07af-47c9-a386-6a90456cc5c0-device-dir\") pod \"aws-ebs-csi-driver-node-wkwv8\" (UID: \"095842a7-07af-47c9-a386-6a90456cc5c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkwv8" Sep 08 08:47:35.986635 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:35.985403 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1746be3a-25ff-4dc5-b4fb-6075f46391e0-original-pull-secret podName:1746be3a-25ff-4dc5-b4fb-6075f46391e0 nodeName:}" failed. No retries permitted until 2026-09-08 08:47:36.485388003 +0000 UTC m=+2.993494764 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1746be3a-25ff-4dc5-b4fb-6075f46391e0-original-pull-secret") pod "global-pull-secret-syncer-b6s4m" (UID: "1746be3a-25ff-4dc5-b4fb-6075f46391e0") : object "kube-system"/"original-pull-secret" not registered Sep 08 08:47:35.986635 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.985408 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-host-run-multus-certs\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.986635 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.985452 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-ovnkube-config\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:35.986635 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.985459 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-multus-daemon-config\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.986635 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.985483 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-cnibin\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.986635 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.985527 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/72a88ec1-2663-4148-9b6f-0b540b3fb77c-agent-certs\") pod \"konnectivity-agent-lmwtf\" (UID: \"72a88ec1-2663-4148-9b6f-0b540b3fb77c\") " pod="kube-system/konnectivity-agent-lmwtf" Sep 08 08:47:35.986635 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.985578 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-multus-socket-dir-parent\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.986635 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.985618 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-host-var-lib-kubelet\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.986635 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.985668 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-hostroot\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.986635 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.985698 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn5zp\" (UniqueName: \"kubernetes.io/projected/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-kube-api-access-cn5zp\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.986635 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.985732 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1746be3a-25ff-4dc5-b4fb-6075f46391e0-dbus\") pod \"global-pull-secret-syncer-b6s4m\" (UID: \"1746be3a-25ff-4dc5-b4fb-6075f46391e0\") " pod="kube-system/global-pull-secret-syncer-b6s4m" Sep 08 08:47:35.986635 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.985761 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/095842a7-07af-47c9-a386-6a90456cc5c0-registration-dir\") pod \"aws-ebs-csi-driver-node-wkwv8\" (UID: \"095842a7-07af-47c9-a386-6a90456cc5c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkwv8" Sep 08 08:47:35.987393 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.985836 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/01944a8d-4aa3-4e4b-9df4-922d4b99f848-serviceca\") pod \"node-ca-8bx7f\" (UID: \"01944a8d-4aa3-4e4b-9df4-922d4b99f848\") " pod="openshift-image-registry/node-ca-8bx7f" Sep 08 08:47:35.987393 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.985865 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9f7affce-54c6-4ec7-8380-6cae8e0edd91-etc-sysctl-conf\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:35.987393 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.985895 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4e49d642-1176-4297-84fd-d0cef9884bff-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wl97n\" (UID: \"4e49d642-1176-4297-84fd-d0cef9884bff\") " pod="openshift-multus/multus-additional-cni-plugins-wl97n" Sep 08 08:47:35.987393 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.985931 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-host-var-lib-cni-bin\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.987393 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.985962 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/095842a7-07af-47c9-a386-6a90456cc5c0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wkwv8\" (UID: \"095842a7-07af-47c9-a386-6a90456cc5c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkwv8" Sep 08 08:47:35.987393 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.986005 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9f7affce-54c6-4ec7-8380-6cae8e0edd91-etc-sysctl-d\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:35.987393 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.986035 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-etc-openvswitch\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:35.987393 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.986065 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-env-overrides\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:35.987393 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.986166 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-ovn-node-metrics-cert\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:35.987393 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.986194 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpqbq\" (UniqueName: \"kubernetes.io/projected/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-kube-api-access-kpqbq\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:35.987393 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.986223 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4e49d642-1176-4297-84fd-d0cef9884bff-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wl97n\" (UID: \"4e49d642-1176-4297-84fd-d0cef9884bff\") " pod="openshift-multus/multus-additional-cni-plugins-wl97n" Sep 08 08:47:35.987393 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.986252 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-host-cni-bin\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:35.987393 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.986283 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4e49d642-1176-4297-84fd-d0cef9884bff-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wl97n\" (UID: \"4e49d642-1176-4297-84fd-d0cef9884bff\") " pod="openshift-multus/multus-additional-cni-plugins-wl97n" Sep 08 08:47:35.987393 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.986464 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4e49d642-1176-4297-84fd-d0cef9884bff-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wl97n\" (UID: \"4e49d642-1176-4297-84fd-d0cef9884bff\") " pod="openshift-multus/multus-additional-cni-plugins-wl97n" Sep 08 08:47:35.987393 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.986517 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/095842a7-07af-47c9-a386-6a90456cc5c0-registration-dir\") pod \"aws-ebs-csi-driver-node-wkwv8\" (UID: \"095842a7-07af-47c9-a386-6a90456cc5c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkwv8" Sep 08 08:47:35.987393 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.986562 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-host-var-lib-kubelet\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.987393 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.986623 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-host-var-lib-cni-bin\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.988202 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.986692 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-multus-socket-dir-parent\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.988202 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.986803 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/095842a7-07af-47c9-a386-6a90456cc5c0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wkwv8\" (UID: \"095842a7-07af-47c9-a386-6a90456cc5c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkwv8" Sep 08 08:47:35.988202 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.986853 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1746be3a-25ff-4dc5-b4fb-6075f46391e0-dbus\") pod \"global-pull-secret-syncer-b6s4m\" (UID: \"1746be3a-25ff-4dc5-b4fb-6075f46391e0\") " pod="kube-system/global-pull-secret-syncer-b6s4m" Sep 08 08:47:35.988202 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.986763 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-hostroot\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:35.988202 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.987196 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/01944a8d-4aa3-4e4b-9df4-922d4b99f848-serviceca\") pod \"node-ca-8bx7f\" (UID: \"01944a8d-4aa3-4e4b-9df4-922d4b99f848\") " pod="openshift-image-registry/node-ca-8bx7f" Sep 08 08:47:35.988202 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.987347 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4e49d642-1176-4297-84fd-d0cef9884bff-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wl97n\" (UID: \"4e49d642-1176-4297-84fd-d0cef9884bff\") " pod="openshift-multus/multus-additional-cni-plugins-wl97n" Sep 08 08:47:35.988202 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.987460 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4e49d642-1176-4297-84fd-d0cef9884bff-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wl97n\" (UID: \"4e49d642-1176-4297-84fd-d0cef9884bff\") " pod="openshift-multus/multus-additional-cni-plugins-wl97n" Sep 08 08:47:35.988202 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.987802 2554 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 08 08:47:35.988872 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:35.988754 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 08:47:35.988872 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:35.988774 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 08:47:35.988872 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:35.988787 2554 projected.go:196] Error preparing data for projected volume kube-api-access-dq6cj for pod openshift-network-diagnostics/network-check-target-z4smd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 08:47:35.988872 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:35.988842 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e400e6f-9bee-43f4-b6b1-f75ba65c39fc-kube-api-access-dq6cj podName:5e400e6f-9bee-43f4-b6b1-f75ba65c39fc nodeName:}" failed. No retries permitted until 2026-09-08 08:47:36.488828276 +0000 UTC m=+2.996935024 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-dq6cj" (UniqueName: "kubernetes.io/projected/5e400e6f-9bee-43f4-b6b1-f75ba65c39fc-kube-api-access-dq6cj") pod "network-check-target-z4smd" (UID: "5e400e6f-9bee-43f4-b6b1-f75ba65c39fc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 08:47:35.993243 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.993217 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bj2cn\" (UniqueName: \"kubernetes.io/projected/095842a7-07af-47c9-a386-6a90456cc5c0-kube-api-access-bj2cn\") pod \"aws-ebs-csi-driver-node-wkwv8\" (UID: \"095842a7-07af-47c9-a386-6a90456cc5c0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkwv8" Sep 08 08:47:35.993351 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.993217 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmvx2\" (UniqueName: \"kubernetes.io/projected/57716ea8-fa5c-47c2-a186-7a0acbf6ae38-kube-api-access-mmvx2\") pod \"network-metrics-daemon-9xk7m\" (UID: \"57716ea8-fa5c-47c2-a186-7a0acbf6ae38\") " pod="openshift-multus/network-metrics-daemon-9xk7m" Sep 08 08:47:35.993351 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.993218 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ld4l\" (UniqueName: \"kubernetes.io/projected/4e49d642-1176-4297-84fd-d0cef9884bff-kube-api-access-8ld4l\") pod \"multus-additional-cni-plugins-wl97n\" (UID: \"4e49d642-1176-4297-84fd-d0cef9884bff\") " pod="openshift-multus/multus-additional-cni-plugins-wl97n" Sep 08 08:47:35.993351 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.993220 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzzsb\" (UniqueName: \"kubernetes.io/projected/01944a8d-4aa3-4e4b-9df4-922d4b99f848-kube-api-access-jzzsb\") pod \"node-ca-8bx7f\" (UID: \"01944a8d-4aa3-4e4b-9df4-922d4b99f848\") " pod="openshift-image-registry/node-ca-8bx7f" Sep 08 08:47:35.995310 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:35.995280 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn5zp\" (UniqueName: \"kubernetes.io/projected/a2e75b83-e08b-4eee-a253-1d6e4ac0e521-kube-api-access-cn5zp\") pod \"multus-txfv4\" (UID: \"a2e75b83-e08b-4eee-a253-1d6e4ac0e521\") " pod="openshift-multus/multus-txfv4" Sep 08 08:47:36.031123 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.031068 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-177.ec2.internal" event={"ID":"fdf29c3cd2e403aedd9a5d1cadaf161a","Type":"ContainerStarted","Data":"b0fa81a81075bb2e967002e9da65ac61e87089af8f28a9ed3cd3a6ad07a3162e"} Sep 08 08:47:36.032532 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.032494 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-177.ec2.internal" event={"ID":"96d81d290f44eaca1c0dbdec93010d30","Type":"ContainerStarted","Data":"c9f2673d2c833acded65d28efc40b1e8edc385bc76023df777e27050b6197424"} Sep 08 08:47:36.086523 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.086486 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9f7affce-54c6-4ec7-8380-6cae8e0edd91-etc-modprobe-d\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:36.086715 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.086527 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-node-log\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.086715 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.086548 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-host-cni-netd\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.086715 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.086609 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-node-log\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.086715 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.086635 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.086715 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.086655 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-run-ovn\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.086715 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.086675 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-ovnkube-script-lib\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.086715 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.086694 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9f7affce-54c6-4ec7-8380-6cae8e0edd91-etc-sysconfig\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:36.086715 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.086694 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9f7affce-54c6-4ec7-8380-6cae8e0edd91-etc-modprobe-d\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:36.086715 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.086716 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9f7affce-54c6-4ec7-8380-6cae8e0edd91-etc-systemd\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:36.087257 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.086742 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-host-kubelet\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.087257 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.086745 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-host-cni-netd\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.087257 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.086771 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-run-systemd\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.087257 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.086789 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-host-run-ovn-kubernetes\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.087257 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.086816 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwqjn\" (UniqueName: \"kubernetes.io/projected/a4db69aa-e494-477b-8263-f4c7d54f0d71-kube-api-access-wwqjn\") pod \"node-resolver-lbvgv\" (UID: \"a4db69aa-e494-477b-8263-f4c7d54f0d71\") " pod="openshift-dns/node-resolver-lbvgv" Sep 08 08:47:36.087257 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.086820 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.087257 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.086830 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9f7affce-54c6-4ec7-8380-6cae8e0edd91-sys\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:36.087257 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.086861 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-run-systemd\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.087257 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.086864 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9f7affce-54c6-4ec7-8380-6cae8e0edd91-lib-modules\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:36.087257 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.086883 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-host-run-ovn-kubernetes\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.087257 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.086788 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-host-kubelet\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.087257 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.086888 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9f7affce-54c6-4ec7-8380-6cae8e0edd91-etc-tuned\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:36.087257 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.086917 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq5qn\" (UniqueName: \"kubernetes.io/projected/9f7affce-54c6-4ec7-8380-6cae8e0edd91-kube-api-access-nq5qn\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:36.087257 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.086934 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-systemd-units\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.087257 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.086953 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-log-socket\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.087257 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.086989 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a4db69aa-e494-477b-8263-f4c7d54f0d71-hosts-file\") pod \"node-resolver-lbvgv\" (UID: \"a4db69aa-e494-477b-8263-f4c7d54f0d71\") " pod="openshift-dns/node-resolver-lbvgv" Sep 08 08:47:36.087257 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.087025 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2rqz\" (UniqueName: \"kubernetes.io/projected/ab4bc45b-8015-494b-b088-5f699f96acaa-kube-api-access-t2rqz\") pod \"iptables-alerter-nsz5v\" (UID: \"ab4bc45b-8015-494b-b088-5f699f96acaa\") " pod="openshift-network-operator/iptables-alerter-nsz5v" Sep 08 08:47:36.088301 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.087040 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f7affce-54c6-4ec7-8380-6cae8e0edd91-tmp\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:36.088301 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.087084 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-var-lib-openvswitch\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.088301 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.087101 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-ovnkube-config\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.088301 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.087117 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/72a88ec1-2663-4148-9b6f-0b540b3fb77c-agent-certs\") pod \"konnectivity-agent-lmwtf\" (UID: \"72a88ec1-2663-4148-9b6f-0b540b3fb77c\") " pod="kube-system/konnectivity-agent-lmwtf" Sep 08 08:47:36.088301 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.087145 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9f7affce-54c6-4ec7-8380-6cae8e0edd91-etc-sysctl-conf\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:36.088301 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.087170 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9f7affce-54c6-4ec7-8380-6cae8e0edd91-etc-sysctl-d\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:36.088301 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.087229 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9f7affce-54c6-4ec7-8380-6cae8e0edd91-etc-systemd\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:36.088301 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.087264 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-etc-openvswitch\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.088301 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.087290 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-env-overrides\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.088301 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.087300 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9f7affce-54c6-4ec7-8380-6cae8e0edd91-etc-sysctl-d\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:36.088301 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.087306 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-ovn-node-metrics-cert\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.088301 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.087323 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpqbq\" (UniqueName: \"kubernetes.io/projected/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-kube-api-access-kpqbq\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.088301 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.087382 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9f7affce-54c6-4ec7-8380-6cae8e0edd91-sys\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:36.088301 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.087478 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-run-ovn\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.088301 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.087482 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-log-socket\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.088301 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.087519 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-host-cni-bin\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.088301 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.087578 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f7affce-54c6-4ec7-8380-6cae8e0edd91-var-lib-kubelet\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:36.088301 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.087594 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9f7affce-54c6-4ec7-8380-6cae8e0edd91-lib-modules\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:36.088868 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.087603 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-host-run-netns\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.088868 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.088042 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-var-lib-openvswitch\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.088868 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.088066 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-host-run-netns\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.088868 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.088083 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-etc-openvswitch\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.088868 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.088107 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-systemd-units\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.088868 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.088133 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9f7affce-54c6-4ec7-8380-6cae8e0edd91-run\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:36.088868 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.088220 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9f7affce-54c6-4ec7-8380-6cae8e0edd91-etc-sysctl-conf\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:36.088868 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.088231 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ab4bc45b-8015-494b-b088-5f699f96acaa-host-slash\") pod \"iptables-alerter-nsz5v\" (UID: \"ab4bc45b-8015-494b-b088-5f699f96acaa\") " pod="openshift-network-operator/iptables-alerter-nsz5v" Sep 08 08:47:36.088868 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.088266 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-host-slash\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.088868 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.088276 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a4db69aa-e494-477b-8263-f4c7d54f0d71-hosts-file\") pod \"node-resolver-lbvgv\" (UID: \"a4db69aa-e494-477b-8263-f4c7d54f0d71\") " pod="openshift-dns/node-resolver-lbvgv" Sep 08 08:47:36.088868 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.088292 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-run-openvswitch\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.088868 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.088316 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9f7affce-54c6-4ec7-8380-6cae8e0edd91-run\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:36.088868 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.088319 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/72a88ec1-2663-4148-9b6f-0b540b3fb77c-konnectivity-ca\") pod \"konnectivity-agent-lmwtf\" (UID: \"72a88ec1-2663-4148-9b6f-0b540b3fb77c\") " pod="kube-system/konnectivity-agent-lmwtf" Sep 08 08:47:36.088868 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.088378 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ab4bc45b-8015-494b-b088-5f699f96acaa-host-slash\") pod \"iptables-alerter-nsz5v\" (UID: \"ab4bc45b-8015-494b-b088-5f699f96acaa\") " pod="openshift-network-operator/iptables-alerter-nsz5v" Sep 08 08:47:36.088868 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.088379 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-host-cni-bin\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.088868 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.088408 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-host-slash\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.088868 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.088414 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4db69aa-e494-477b-8263-f4c7d54f0d71-tmp-dir\") pod \"node-resolver-lbvgv\" (UID: \"a4db69aa-e494-477b-8263-f4c7d54f0d71\") " pod="openshift-dns/node-resolver-lbvgv" Sep 08 08:47:36.088868 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.088445 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ab4bc45b-8015-494b-b088-5f699f96acaa-iptables-alerter-script\") pod \"iptables-alerter-nsz5v\" (UID: \"ab4bc45b-8015-494b-b088-5f699f96acaa\") " pod="openshift-network-operator/iptables-alerter-nsz5v" Sep 08 08:47:36.089740 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.088469 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f7affce-54c6-4ec7-8380-6cae8e0edd91-etc-kubernetes\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:36.089740 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.088493 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9f7affce-54c6-4ec7-8380-6cae8e0edd91-host\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:36.089740 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.088561 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-env-overrides\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.089740 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.088609 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9f7affce-54c6-4ec7-8380-6cae8e0edd91-host\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:36.089740 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.088613 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9f7affce-54c6-4ec7-8380-6cae8e0edd91-etc-sysconfig\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:36.089740 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.088698 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-ovnkube-script-lib\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.089740 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.088888 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4db69aa-e494-477b-8263-f4c7d54f0d71-tmp-dir\") pod \"node-resolver-lbvgv\" (UID: \"a4db69aa-e494-477b-8263-f4c7d54f0d71\") " pod="openshift-dns/node-resolver-lbvgv" Sep 08 08:47:36.089740 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.088924 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-ovnkube-config\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.089740 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.088998 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f7affce-54c6-4ec7-8380-6cae8e0edd91-etc-kubernetes\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:36.089740 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.089006 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f7affce-54c6-4ec7-8380-6cae8e0edd91-var-lib-kubelet\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:36.089740 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.089037 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-run-openvswitch\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.089740 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.089094 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/72a88ec1-2663-4148-9b6f-0b540b3fb77c-konnectivity-ca\") pod \"konnectivity-agent-lmwtf\" (UID: \"72a88ec1-2663-4148-9b6f-0b540b3fb77c\") " pod="kube-system/konnectivity-agent-lmwtf" Sep 08 08:47:36.090337 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.090025 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ab4bc45b-8015-494b-b088-5f699f96acaa-iptables-alerter-script\") pod \"iptables-alerter-nsz5v\" (UID: \"ab4bc45b-8015-494b-b088-5f699f96acaa\") " pod="openshift-network-operator/iptables-alerter-nsz5v" Sep 08 08:47:36.090337 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.090322 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-ovn-node-metrics-cert\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.090439 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.090415 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9f7affce-54c6-4ec7-8380-6cae8e0edd91-etc-tuned\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:36.090491 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.090436 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/72a88ec1-2663-4148-9b6f-0b540b3fb77c-agent-certs\") pod \"konnectivity-agent-lmwtf\" (UID: \"72a88ec1-2663-4148-9b6f-0b540b3fb77c\") " pod="kube-system/konnectivity-agent-lmwtf" Sep 08 08:47:36.090808 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.090789 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f7affce-54c6-4ec7-8380-6cae8e0edd91-tmp\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:36.096242 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.096200 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwqjn\" (UniqueName: \"kubernetes.io/projected/a4db69aa-e494-477b-8263-f4c7d54f0d71-kube-api-access-wwqjn\") pod \"node-resolver-lbvgv\" (UID: \"a4db69aa-e494-477b-8263-f4c7d54f0d71\") " pod="openshift-dns/node-resolver-lbvgv" Sep 08 08:47:36.096414 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.096393 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nq5qn\" (UniqueName: \"kubernetes.io/projected/9f7affce-54c6-4ec7-8380-6cae8e0edd91-kube-api-access-nq5qn\") pod \"tuned-gxjvt\" (UID: \"9f7affce-54c6-4ec7-8380-6cae8e0edd91\") " pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:36.096682 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.096645 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2rqz\" (UniqueName: \"kubernetes.io/projected/ab4bc45b-8015-494b-b088-5f699f96acaa-kube-api-access-t2rqz\") pod \"iptables-alerter-nsz5v\" (UID: \"ab4bc45b-8015-494b-b088-5f699f96acaa\") " pod="openshift-network-operator/iptables-alerter-nsz5v" Sep 08 08:47:36.097095 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.097075 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpqbq\" (UniqueName: \"kubernetes.io/projected/04c4d313-ed7e-4fea-bec5-3ed11857a8c4-kube-api-access-kpqbq\") pod \"ovnkube-node-xn2lh\" (UID: \"04c4d313-ed7e-4fea-bec5-3ed11857a8c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.172149 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.172082 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8bx7f" Sep 08 08:47:36.179447 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.179426 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wl97n" Sep 08 08:47:36.193488 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.193468 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-txfv4" Sep 08 08:47:36.199151 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.199130 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkwv8" Sep 08 08:47:36.207091 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.207073 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nsz5v" Sep 08 08:47:36.214791 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.214772 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:47:36.222517 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.222498 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lmwtf" Sep 08 08:47:36.229233 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.229216 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" Sep 08 08:47:36.235850 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.235826 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lbvgv" Sep 08 08:47:36.490956 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.490926 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1746be3a-25ff-4dc5-b4fb-6075f46391e0-original-pull-secret\") pod \"global-pull-secret-syncer-b6s4m\" (UID: \"1746be3a-25ff-4dc5-b4fb-6075f46391e0\") " pod="kube-system/global-pull-secret-syncer-b6s4m" Sep 08 08:47:36.491139 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.490994 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57716ea8-fa5c-47c2-a186-7a0acbf6ae38-metrics-certs\") pod \"network-metrics-daemon-9xk7m\" (UID: \"57716ea8-fa5c-47c2-a186-7a0acbf6ae38\") " pod="openshift-multus/network-metrics-daemon-9xk7m" Sep 08 08:47:36.491139 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.491021 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dq6cj\" (UniqueName: \"kubernetes.io/projected/5e400e6f-9bee-43f4-b6b1-f75ba65c39fc-kube-api-access-dq6cj\") pod \"network-check-target-z4smd\" (UID: \"5e400e6f-9bee-43f4-b6b1-f75ba65c39fc\") " pod="openshift-network-diagnostics/network-check-target-z4smd" Sep 08 08:47:36.491139 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:36.491084 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 08:47:36.491139 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:36.491133 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 08:47:36.491312 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:36.491146 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1746be3a-25ff-4dc5-b4fb-6075f46391e0-original-pull-secret podName:1746be3a-25ff-4dc5-b4fb-6075f46391e0 nodeName:}" failed. No retries permitted until 2026-09-08 08:47:37.491131272 +0000 UTC m=+3.999238018 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1746be3a-25ff-4dc5-b4fb-6075f46391e0-original-pull-secret") pod "global-pull-secret-syncer-b6s4m" (UID: "1746be3a-25ff-4dc5-b4fb-6075f46391e0") : object "kube-system"/"original-pull-secret" not registered Sep 08 08:47:36.491312 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:36.491146 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 08:47:36.491312 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:36.491166 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 08:47:36.491312 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:36.491179 2554 projected.go:196] Error preparing data for projected volume kube-api-access-dq6cj for pod openshift-network-diagnostics/network-check-target-z4smd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 08:47:36.491312 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:36.491181 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57716ea8-fa5c-47c2-a186-7a0acbf6ae38-metrics-certs podName:57716ea8-fa5c-47c2-a186-7a0acbf6ae38 nodeName:}" failed. No retries permitted until 2026-09-08 08:47:37.491169032 +0000 UTC m=+3.999275778 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57716ea8-fa5c-47c2-a186-7a0acbf6ae38-metrics-certs") pod "network-metrics-daemon-9xk7m" (UID: "57716ea8-fa5c-47c2-a186-7a0acbf6ae38") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 08:47:36.491312 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:36.491219 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e400e6f-9bee-43f4-b6b1-f75ba65c39fc-kube-api-access-dq6cj podName:5e400e6f-9bee-43f4-b6b1-f75ba65c39fc nodeName:}" failed. No retries permitted until 2026-09-08 08:47:37.491206497 +0000 UTC m=+3.999313245 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-dq6cj" (UniqueName: "kubernetes.io/projected/5e400e6f-9bee-43f4-b6b1-f75ba65c39fc-kube-api-access-dq6cj") pod "network-check-target-z4smd" (UID: "5e400e6f-9bee-43f4-b6b1-f75ba65c39fc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 08:47:36.912476 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.912391 2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 08:42:35 +0000 UTC" deadline="2028-04-22 00:31:41.687186375 +0000 UTC" Sep 08 08:47:36.912476 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:36.912430 2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14199h44m4.774761469s" Sep 08 08:47:37.027489 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:37.027451 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4smd" Sep 08 08:47:37.027664 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:37.027498 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xk7m" Sep 08 08:47:37.027664 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:37.027583 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4smd" podUID="5e400e6f-9bee-43f4-b6b1-f75ba65c39fc" Sep 08 08:47:37.027759 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:37.027732 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xk7m" podUID="57716ea8-fa5c-47c2-a186-7a0acbf6ae38" Sep 08 08:47:37.154323 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:37.154262 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab4bc45b_8015_494b_b088_5f699f96acaa.slice/crio-b5ecdf5701b14956bd60cc7efa654cd45e7e3eb4d6d9940a47da0c123cb61a65 WatchSource:0}: Error finding container b5ecdf5701b14956bd60cc7efa654cd45e7e3eb4d6d9940a47da0c123cb61a65: Status 404 returned error can't find the container with id b5ecdf5701b14956bd60cc7efa654cd45e7e3eb4d6d9940a47da0c123cb61a65 Sep 08 08:47:37.155760 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:37.155734 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e49d642_1176_4297_84fd_d0cef9884bff.slice/crio-28679b32d774ae715136e6e1b13083162ef41fc4c9d3156113d49d2cf96a2954 WatchSource:0}: Error finding container 28679b32d774ae715136e6e1b13083162ef41fc4c9d3156113d49d2cf96a2954: Status 404 returned error can't find the container with id 28679b32d774ae715136e6e1b13083162ef41fc4c9d3156113d49d2cf96a2954 Sep 08 08:47:37.156350 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:37.156321 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72a88ec1_2663_4148_9b6f_0b540b3fb77c.slice/crio-9a448999e895bfd6016fad17deb0e37b38b05412cc0b98a33bf7146713e10926 WatchSource:0}: Error finding container 9a448999e895bfd6016fad17deb0e37b38b05412cc0b98a33bf7146713e10926: Status 404 returned error can't find the container with id 9a448999e895bfd6016fad17deb0e37b38b05412cc0b98a33bf7146713e10926 Sep 08 08:47:37.159602 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:37.159581 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01944a8d_4aa3_4e4b_9df4_922d4b99f848.slice/crio-bbe456004f546ca9f1ca8bdbf76c9a2f73335fac3f1cf5417bd2c154890eb3da WatchSource:0}: Error finding container bbe456004f546ca9f1ca8bdbf76c9a2f73335fac3f1cf5417bd2c154890eb3da: Status 404 returned error can't find the container with id bbe456004f546ca9f1ca8bdbf76c9a2f73335fac3f1cf5417bd2c154890eb3da Sep 08 08:47:37.160590 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:47:37.160570 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod095842a7_07af_47c9_a386_6a90456cc5c0.slice/crio-6c5556d834ed7249d09c6beb7a2a890fcbff3d470bc8a0dcdbba9dfadff55541 WatchSource:0}: Error finding container 6c5556d834ed7249d09c6beb7a2a890fcbff3d470bc8a0dcdbba9dfadff55541: Status 404 returned error can't find the container with id 6c5556d834ed7249d09c6beb7a2a890fcbff3d470bc8a0dcdbba9dfadff55541 Sep 08 08:47:37.500270 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:37.500237 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1746be3a-25ff-4dc5-b4fb-6075f46391e0-original-pull-secret\") pod \"global-pull-secret-syncer-b6s4m\" (UID: \"1746be3a-25ff-4dc5-b4fb-6075f46391e0\") " pod="kube-system/global-pull-secret-syncer-b6s4m" Sep 08 08:47:37.500383 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:37.500281 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57716ea8-fa5c-47c2-a186-7a0acbf6ae38-metrics-certs\") pod \"network-metrics-daemon-9xk7m\" (UID: \"57716ea8-fa5c-47c2-a186-7a0acbf6ae38\") " pod="openshift-multus/network-metrics-daemon-9xk7m" Sep 08 08:47:37.500383 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:37.500300 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dq6cj\" (UniqueName: \"kubernetes.io/projected/5e400e6f-9bee-43f4-b6b1-f75ba65c39fc-kube-api-access-dq6cj\") pod \"network-check-target-z4smd\" (UID: \"5e400e6f-9bee-43f4-b6b1-f75ba65c39fc\") " pod="openshift-network-diagnostics/network-check-target-z4smd" Sep 08 08:47:37.500383 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:37.500375 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 08:47:37.500497 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:37.500397 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 08:47:37.500497 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:37.500411 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 08:47:37.500497 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:37.500420 2554 projected.go:196] Error preparing data for projected volume kube-api-access-dq6cj for pod openshift-network-diagnostics/network-check-target-z4smd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 08:47:37.500497 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:37.500431 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57716ea8-fa5c-47c2-a186-7a0acbf6ae38-metrics-certs podName:57716ea8-fa5c-47c2-a186-7a0acbf6ae38 nodeName:}" failed. No retries permitted until 2026-09-08 08:47:39.500418048 +0000 UTC m=+6.008524793 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57716ea8-fa5c-47c2-a186-7a0acbf6ae38-metrics-certs") pod "network-metrics-daemon-9xk7m" (UID: "57716ea8-fa5c-47c2-a186-7a0acbf6ae38") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 08:47:37.500497 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:37.500452 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e400e6f-9bee-43f4-b6b1-f75ba65c39fc-kube-api-access-dq6cj podName:5e400e6f-9bee-43f4-b6b1-f75ba65c39fc nodeName:}" failed. No retries permitted until 2026-09-08 08:47:39.50044212 +0000 UTC m=+6.008548866 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-dq6cj" (UniqueName: "kubernetes.io/projected/5e400e6f-9bee-43f4-b6b1-f75ba65c39fc-kube-api-access-dq6cj") pod "network-check-target-z4smd" (UID: "5e400e6f-9bee-43f4-b6b1-f75ba65c39fc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 08:47:37.500497 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:37.500374 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 08:47:37.500497 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:37.500498 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1746be3a-25ff-4dc5-b4fb-6075f46391e0-original-pull-secret podName:1746be3a-25ff-4dc5-b4fb-6075f46391e0 nodeName:}" failed. No retries permitted until 2026-09-08 08:47:39.500486708 +0000 UTC m=+6.008593457 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1746be3a-25ff-4dc5-b4fb-6075f46391e0-original-pull-secret") pod "global-pull-secret-syncer-b6s4m" (UID: "1746be3a-25ff-4dc5-b4fb-6075f46391e0") : object "kube-system"/"original-pull-secret" not registered Sep 08 08:47:37.913008 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:37.912854 2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 08:42:35 +0000 UTC" deadline="2028-02-08 20:10:38.738930999 +0000 UTC" Sep 08 08:47:37.913008 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:37.912912 2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12443h23m0.826022978s" Sep 08 08:47:38.027854 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:38.027383 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6s4m" Sep 08 08:47:38.027854 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:38.027509 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6s4m" podUID="1746be3a-25ff-4dc5-b4fb-6075f46391e0" Sep 08 08:47:38.040931 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:38.040864 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-txfv4" event={"ID":"a2e75b83-e08b-4eee-a253-1d6e4ac0e521","Type":"ContainerStarted","Data":"f032eb21d85b6f3c1164169b99cf1ce69636a258e21898f4d4c09442122b6e76"} Sep 08 08:47:38.052655 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:38.052587 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" event={"ID":"9f7affce-54c6-4ec7-8380-6cae8e0edd91","Type":"ContainerStarted","Data":"e54c2698c254e017a63cad695eb03d07f20d018df53f900d53866f991a2312a5"} Sep 08 08:47:38.062940 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:38.062845 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" event={"ID":"04c4d313-ed7e-4fea-bec5-3ed11857a8c4","Type":"ContainerStarted","Data":"5e111cd7f7aece751ac102b822cb3a3b19f84c1bf144b718706d69017d80b516"} Sep 08 08:47:38.069727 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:38.069702 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8bx7f" event={"ID":"01944a8d-4aa3-4e4b-9df4-922d4b99f848","Type":"ContainerStarted","Data":"bbe456004f546ca9f1ca8bdbf76c9a2f73335fac3f1cf5417bd2c154890eb3da"} Sep 08 08:47:38.090096 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:38.090032 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lmwtf" event={"ID":"72a88ec1-2663-4148-9b6f-0b540b3fb77c","Type":"ContainerStarted","Data":"9a448999e895bfd6016fad17deb0e37b38b05412cc0b98a33bf7146713e10926"} Sep 08 08:47:38.098834 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:38.098810 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wl97n" event={"ID":"4e49d642-1176-4297-84fd-d0cef9884bff","Type":"ContainerStarted","Data":"28679b32d774ae715136e6e1b13083162ef41fc4c9d3156113d49d2cf96a2954"} Sep 08 08:47:38.102658 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:38.102595 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nsz5v" event={"ID":"ab4bc45b-8015-494b-b088-5f699f96acaa","Type":"ContainerStarted","Data":"b5ecdf5701b14956bd60cc7efa654cd45e7e3eb4d6d9940a47da0c123cb61a65"} Sep 08 08:47:38.110628 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:38.109994 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-177.ec2.internal" event={"ID":"fdf29c3cd2e403aedd9a5d1cadaf161a","Type":"ContainerStarted","Data":"263ad98eb91b2d158b4a08d6ab1abb8753452ee2e799efb54726794088ddab1d"} Sep 08 08:47:38.116968 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:38.116924 2554 generic.go:356] "Generic (PLEG): container finished" podID="96d81d290f44eaca1c0dbdec93010d30" containerID="43984f33befc6e95c79ea51dc8b61cbe7c7b33f0377c4cc3bfa1a7ac2f1c8fad" exitCode=0 Sep 08 08:47:38.117075 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:38.117011 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-177.ec2.internal" event={"ID":"96d81d290f44eaca1c0dbdec93010d30","Type":"ContainerDied","Data":"43984f33befc6e95c79ea51dc8b61cbe7c7b33f0377c4cc3bfa1a7ac2f1c8fad"} Sep 08 08:47:38.124745 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:38.124718 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lbvgv" event={"ID":"a4db69aa-e494-477b-8263-f4c7d54f0d71","Type":"ContainerStarted","Data":"50a28a50e16fc72013e3d243db1de7d7e54ed1463bf55dcf5d7778e544baff08"} Sep 08 08:47:38.124865 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:38.124765 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-177.ec2.internal" podStartSLOduration=4.124748854 podStartE2EDuration="4.124748854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 08:47:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:47:38.124489168 +0000 UTC m=+4.632595937" watchObservedRunningTime="2026-09-08 08:47:38.124748854 +0000 UTC m=+4.632855624" Sep 08 08:47:38.135911 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:38.135883 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkwv8" event={"ID":"095842a7-07af-47c9-a386-6a90456cc5c0","Type":"ContainerStarted","Data":"6c5556d834ed7249d09c6beb7a2a890fcbff3d470bc8a0dcdbba9dfadff55541"} Sep 08 08:47:39.027346 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:39.027098 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xk7m" Sep 08 08:47:39.027784 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:39.027454 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xk7m" podUID="57716ea8-fa5c-47c2-a186-7a0acbf6ae38" Sep 08 08:47:39.027931 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:39.027912 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4smd" Sep 08 08:47:39.028046 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:39.028025 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4smd" podUID="5e400e6f-9bee-43f4-b6b1-f75ba65c39fc" Sep 08 08:47:39.157481 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:39.157442 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-177.ec2.internal" event={"ID":"96d81d290f44eaca1c0dbdec93010d30","Type":"ContainerStarted","Data":"1b3ec89ec44346b02ce1dc55e659e93275b5c8e2dc2a3854fb14ca7813e2090b"} Sep 08 08:47:39.519011 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:39.518957 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1746be3a-25ff-4dc5-b4fb-6075f46391e0-original-pull-secret\") pod \"global-pull-secret-syncer-b6s4m\" (UID: \"1746be3a-25ff-4dc5-b4fb-6075f46391e0\") " pod="kube-system/global-pull-secret-syncer-b6s4m" Sep 08 08:47:39.519208 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:39.519047 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57716ea8-fa5c-47c2-a186-7a0acbf6ae38-metrics-certs\") pod \"network-metrics-daemon-9xk7m\" (UID: \"57716ea8-fa5c-47c2-a186-7a0acbf6ae38\") " pod="openshift-multus/network-metrics-daemon-9xk7m" Sep 08 08:47:39.519208 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:39.519075 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dq6cj\" (UniqueName: \"kubernetes.io/projected/5e400e6f-9bee-43f4-b6b1-f75ba65c39fc-kube-api-access-dq6cj\") pod \"network-check-target-z4smd\" (UID: \"5e400e6f-9bee-43f4-b6b1-f75ba65c39fc\") " pod="openshift-network-diagnostics/network-check-target-z4smd" Sep 08 08:47:39.519321 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:39.519236 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 08:47:39.519321 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:39.519258 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 08:47:39.519321 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:39.519271 2554 projected.go:196] Error preparing data for projected volume kube-api-access-dq6cj for pod openshift-network-diagnostics/network-check-target-z4smd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 08:47:39.519490 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:39.519329 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e400e6f-9bee-43f4-b6b1-f75ba65c39fc-kube-api-access-dq6cj podName:5e400e6f-9bee-43f4-b6b1-f75ba65c39fc nodeName:}" failed. No retries permitted until 2026-09-08 08:47:43.519310218 +0000 UTC m=+10.027416969 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-dq6cj" (UniqueName: "kubernetes.io/projected/5e400e6f-9bee-43f4-b6b1-f75ba65c39fc-kube-api-access-dq6cj") pod "network-check-target-z4smd" (UID: "5e400e6f-9bee-43f4-b6b1-f75ba65c39fc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 08:47:39.519727 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:39.519708 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 08:47:39.519798 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:39.519769 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1746be3a-25ff-4dc5-b4fb-6075f46391e0-original-pull-secret podName:1746be3a-25ff-4dc5-b4fb-6075f46391e0 nodeName:}" failed. No retries permitted until 2026-09-08 08:47:43.519746396 +0000 UTC m=+10.027853149 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1746be3a-25ff-4dc5-b4fb-6075f46391e0-original-pull-secret") pod "global-pull-secret-syncer-b6s4m" (UID: "1746be3a-25ff-4dc5-b4fb-6075f46391e0") : object "kube-system"/"original-pull-secret" not registered Sep 08 08:47:39.519860 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:39.519845 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 08:47:39.519912 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:39.519883 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57716ea8-fa5c-47c2-a186-7a0acbf6ae38-metrics-certs podName:57716ea8-fa5c-47c2-a186-7a0acbf6ae38 nodeName:}" failed. No retries permitted until 2026-09-08 08:47:43.51986989 +0000 UTC m=+10.027976640 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57716ea8-fa5c-47c2-a186-7a0acbf6ae38-metrics-certs") pod "network-metrics-daemon-9xk7m" (UID: "57716ea8-fa5c-47c2-a186-7a0acbf6ae38") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 08:47:40.027188 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:40.027158 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6s4m" Sep 08 08:47:40.027364 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:40.027280 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6s4m" podUID="1746be3a-25ff-4dc5-b4fb-6075f46391e0" Sep 08 08:47:41.027213 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:41.027182 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4smd" Sep 08 08:47:41.027391 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:41.027299 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4smd" podUID="5e400e6f-9bee-43f4-b6b1-f75ba65c39fc" Sep 08 08:47:41.027391 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:41.027337 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xk7m" Sep 08 08:47:41.027821 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:41.027455 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xk7m" podUID="57716ea8-fa5c-47c2-a186-7a0acbf6ae38" Sep 08 08:47:42.027048 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:42.027020 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6s4m" Sep 08 08:47:42.027219 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:42.027147 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6s4m" podUID="1746be3a-25ff-4dc5-b4fb-6075f46391e0" Sep 08 08:47:43.027754 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:43.027138 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xk7m" Sep 08 08:47:43.027754 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:43.027283 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xk7m" podUID="57716ea8-fa5c-47c2-a186-7a0acbf6ae38" Sep 08 08:47:43.027754 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:43.027629 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4smd" Sep 08 08:47:43.027754 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:43.027710 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4smd" podUID="5e400e6f-9bee-43f4-b6b1-f75ba65c39fc" Sep 08 08:47:43.553750 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:43.553708 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1746be3a-25ff-4dc5-b4fb-6075f46391e0-original-pull-secret\") pod \"global-pull-secret-syncer-b6s4m\" (UID: \"1746be3a-25ff-4dc5-b4fb-6075f46391e0\") " pod="kube-system/global-pull-secret-syncer-b6s4m" Sep 08 08:47:43.553933 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:43.553770 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57716ea8-fa5c-47c2-a186-7a0acbf6ae38-metrics-certs\") pod \"network-metrics-daemon-9xk7m\" (UID: \"57716ea8-fa5c-47c2-a186-7a0acbf6ae38\") " pod="openshift-multus/network-metrics-daemon-9xk7m" Sep 08 08:47:43.553933 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:43.553790 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dq6cj\" (UniqueName: \"kubernetes.io/projected/5e400e6f-9bee-43f4-b6b1-f75ba65c39fc-kube-api-access-dq6cj\") pod \"network-check-target-z4smd\" (UID: \"5e400e6f-9bee-43f4-b6b1-f75ba65c39fc\") " pod="openshift-network-diagnostics/network-check-target-z4smd" Sep 08 08:47:43.553933 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:43.553879 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 08:47:43.553933 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:43.553900 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 08:47:43.553933 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:43.553913 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 08:47:43.553933 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:43.553921 2554 projected.go:196] Error preparing data for projected volume kube-api-access-dq6cj for pod openshift-network-diagnostics/network-check-target-z4smd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 08:47:43.554262 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:43.553939 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 08:47:43.554262 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:43.553943 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1746be3a-25ff-4dc5-b4fb-6075f46391e0-original-pull-secret podName:1746be3a-25ff-4dc5-b4fb-6075f46391e0 nodeName:}" failed. No retries permitted until 2026-09-08 08:47:51.553923591 +0000 UTC m=+18.062030353 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1746be3a-25ff-4dc5-b4fb-6075f46391e0-original-pull-secret") pod "global-pull-secret-syncer-b6s4m" (UID: "1746be3a-25ff-4dc5-b4fb-6075f46391e0") : object "kube-system"/"original-pull-secret" not registered Sep 08 08:47:43.554262 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:43.554005 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e400e6f-9bee-43f4-b6b1-f75ba65c39fc-kube-api-access-dq6cj podName:5e400e6f-9bee-43f4-b6b1-f75ba65c39fc nodeName:}" failed. No retries permitted until 2026-09-08 08:47:51.553991002 +0000 UTC m=+18.062097754 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-dq6cj" (UniqueName: "kubernetes.io/projected/5e400e6f-9bee-43f4-b6b1-f75ba65c39fc-kube-api-access-dq6cj") pod "network-check-target-z4smd" (UID: "5e400e6f-9bee-43f4-b6b1-f75ba65c39fc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 08:47:43.554262 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:43.554019 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57716ea8-fa5c-47c2-a186-7a0acbf6ae38-metrics-certs podName:57716ea8-fa5c-47c2-a186-7a0acbf6ae38 nodeName:}" failed. No retries permitted until 2026-09-08 08:47:51.554011501 +0000 UTC m=+18.062118254 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57716ea8-fa5c-47c2-a186-7a0acbf6ae38-metrics-certs") pod "network-metrics-daemon-9xk7m" (UID: "57716ea8-fa5c-47c2-a186-7a0acbf6ae38") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 08:47:44.028792 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:44.028759 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6s4m" Sep 08 08:47:44.029250 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:44.028909 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6s4m" podUID="1746be3a-25ff-4dc5-b4fb-6075f46391e0" Sep 08 08:47:45.026760 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:45.026719 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xk7m" Sep 08 08:47:45.026938 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:45.026872 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xk7m" podUID="57716ea8-fa5c-47c2-a186-7a0acbf6ae38" Sep 08 08:47:45.027071 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:45.026725 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4smd" Sep 08 08:47:45.027071 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:45.027046 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4smd" podUID="5e400e6f-9bee-43f4-b6b1-f75ba65c39fc" Sep 08 08:47:46.027396 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:46.027366 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6s4m" Sep 08 08:47:46.027847 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:46.027546 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6s4m" podUID="1746be3a-25ff-4dc5-b4fb-6075f46391e0" Sep 08 08:47:47.027213 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:47.027181 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4smd" Sep 08 08:47:47.027388 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:47.027226 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xk7m" Sep 08 08:47:47.027388 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:47.027300 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4smd" podUID="5e400e6f-9bee-43f4-b6b1-f75ba65c39fc" Sep 08 08:47:47.027770 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:47.027409 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xk7m" podUID="57716ea8-fa5c-47c2-a186-7a0acbf6ae38" Sep 08 08:47:48.027224 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:48.027194 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6s4m" Sep 08 08:47:48.027410 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:48.027319 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6s4m" podUID="1746be3a-25ff-4dc5-b4fb-6075f46391e0" Sep 08 08:47:49.027524 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:49.027487 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4smd" Sep 08 08:47:49.028002 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:49.027597 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4smd" podUID="5e400e6f-9bee-43f4-b6b1-f75ba65c39fc" Sep 08 08:47:49.028002 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:49.027668 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xk7m" Sep 08 08:47:49.028002 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:49.027795 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xk7m" podUID="57716ea8-fa5c-47c2-a186-7a0acbf6ae38" Sep 08 08:47:50.026560 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:50.026518 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6s4m" Sep 08 08:47:50.026737 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:50.026650 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6s4m" podUID="1746be3a-25ff-4dc5-b4fb-6075f46391e0" Sep 08 08:47:51.027093 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:51.027058 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4smd" Sep 08 08:47:51.027568 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:51.027168 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4smd" podUID="5e400e6f-9bee-43f4-b6b1-f75ba65c39fc" Sep 08 08:47:51.027568 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:51.027213 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xk7m" Sep 08 08:47:51.027568 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:51.027301 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xk7m" podUID="57716ea8-fa5c-47c2-a186-7a0acbf6ae38" Sep 08 08:47:51.617450 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:51.617420 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57716ea8-fa5c-47c2-a186-7a0acbf6ae38-metrics-certs\") pod \"network-metrics-daemon-9xk7m\" (UID: \"57716ea8-fa5c-47c2-a186-7a0acbf6ae38\") " pod="openshift-multus/network-metrics-daemon-9xk7m" Sep 08 08:47:51.617450 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:51.617454 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dq6cj\" (UniqueName: \"kubernetes.io/projected/5e400e6f-9bee-43f4-b6b1-f75ba65c39fc-kube-api-access-dq6cj\") pod \"network-check-target-z4smd\" (UID: \"5e400e6f-9bee-43f4-b6b1-f75ba65c39fc\") " pod="openshift-network-diagnostics/network-check-target-z4smd" Sep 08 08:47:51.617679 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:51.617580 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1746be3a-25ff-4dc5-b4fb-6075f46391e0-original-pull-secret\") pod \"global-pull-secret-syncer-b6s4m\" (UID: \"1746be3a-25ff-4dc5-b4fb-6075f46391e0\") " pod="kube-system/global-pull-secret-syncer-b6s4m" Sep 08 08:47:51.617679 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:51.617583 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 08:47:51.617679 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:51.617594 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 08:47:51.617824 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:51.617691 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 08:47:51.617824 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:51.617706 2554 projected.go:196] Error preparing data for projected volume kube-api-access-dq6cj for pod openshift-network-diagnostics/network-check-target-z4smd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 08:47:51.617824 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:51.617708 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 08:47:51.617824 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:51.617664 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57716ea8-fa5c-47c2-a186-7a0acbf6ae38-metrics-certs podName:57716ea8-fa5c-47c2-a186-7a0acbf6ae38 nodeName:}" failed. No retries permitted until 2026-09-08 08:48:07.617648491 +0000 UTC m=+34.125755243 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57716ea8-fa5c-47c2-a186-7a0acbf6ae38-metrics-certs") pod "network-metrics-daemon-9xk7m" (UID: "57716ea8-fa5c-47c2-a186-7a0acbf6ae38") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 08:47:51.617824 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:51.617770 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e400e6f-9bee-43f4-b6b1-f75ba65c39fc-kube-api-access-dq6cj podName:5e400e6f-9bee-43f4-b6b1-f75ba65c39fc nodeName:}" failed. No retries permitted until 2026-09-08 08:48:07.61775242 +0000 UTC m=+34.125859178 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-dq6cj" (UniqueName: "kubernetes.io/projected/5e400e6f-9bee-43f4-b6b1-f75ba65c39fc-kube-api-access-dq6cj") pod "network-check-target-z4smd" (UID: "5e400e6f-9bee-43f4-b6b1-f75ba65c39fc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 08:47:51.617824 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:51.617792 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1746be3a-25ff-4dc5-b4fb-6075f46391e0-original-pull-secret podName:1746be3a-25ff-4dc5-b4fb-6075f46391e0 nodeName:}" failed. No retries permitted until 2026-09-08 08:48:07.617783103 +0000 UTC m=+34.125889855 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1746be3a-25ff-4dc5-b4fb-6075f46391e0-original-pull-secret") pod "global-pull-secret-syncer-b6s4m" (UID: "1746be3a-25ff-4dc5-b4fb-6075f46391e0") : object "kube-system"/"original-pull-secret" not registered Sep 08 08:47:52.027522 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:52.027491 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6s4m" Sep 08 08:47:52.028024 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:52.027613 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6s4m" podUID="1746be3a-25ff-4dc5-b4fb-6075f46391e0" Sep 08 08:47:53.026836 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:53.026807 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4smd" Sep 08 08:47:53.027032 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:53.026808 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xk7m" Sep 08 08:47:53.027032 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:53.026927 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4smd" podUID="5e400e6f-9bee-43f4-b6b1-f75ba65c39fc" Sep 08 08:47:53.027117 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:53.027057 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xk7m" podUID="57716ea8-fa5c-47c2-a186-7a0acbf6ae38" Sep 08 08:47:54.027282 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:54.027253 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6s4m" Sep 08 08:47:54.027670 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:54.027341 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6s4m" podUID="1746be3a-25ff-4dc5-b4fb-6075f46391e0" Sep 08 08:47:55.027485 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:55.027330 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4smd" Sep 08 08:47:55.027868 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:55.027342 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xk7m" Sep 08 08:47:55.027868 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:55.027599 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4smd" podUID="5e400e6f-9bee-43f4-b6b1-f75ba65c39fc" Sep 08 08:47:55.027868 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:55.027699 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xk7m" podUID="57716ea8-fa5c-47c2-a186-7a0acbf6ae38" Sep 08 08:47:55.194508 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:55.194472 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-txfv4" event={"ID":"a2e75b83-e08b-4eee-a253-1d6e4ac0e521","Type":"ContainerStarted","Data":"9cb09dc460756b12da89071dbba5ae9fbd0b0fc7a8faaa02d744ffff26da711a"} Sep 08 08:47:55.196042 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:55.195945 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" event={"ID":"9f7affce-54c6-4ec7-8380-6cae8e0edd91","Type":"ContainerStarted","Data":"b87ea68d9d7939f3b2d72dc81963a5d1543fd44ee8f245df3aa15dbf52b47a5d"} Sep 08 08:47:55.197608 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:55.197590 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xn2lh_04c4d313-ed7e-4fea-bec5-3ed11857a8c4/ovn-acl-logging/0.log" Sep 08 08:47:55.197993 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:55.197962 2554 generic.go:356] "Generic (PLEG): container finished" podID="04c4d313-ed7e-4fea-bec5-3ed11857a8c4" containerID="655d77035a31db3f450fa709f437e625009c02cebf7d27bf8f2431d66212bf1b" exitCode=1 Sep 08 08:47:55.198100 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:55.198004 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" event={"ID":"04c4d313-ed7e-4fea-bec5-3ed11857a8c4","Type":"ContainerDied","Data":"655d77035a31db3f450fa709f437e625009c02cebf7d27bf8f2431d66212bf1b"} Sep 08 08:47:55.198100 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:55.198063 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" event={"ID":"04c4d313-ed7e-4fea-bec5-3ed11857a8c4","Type":"ContainerStarted","Data":"c80185a46f3f3511d4520c867702633957faea86a2afc5d36f9956bfeef27ca0"} Sep 08 08:47:55.199509 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:55.199488 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8bx7f" event={"ID":"01944a8d-4aa3-4e4b-9df4-922d4b99f848","Type":"ContainerStarted","Data":"05b48f44ef075cff3fdb217599f02e008eb44bf2d06122b89f8ac14a517dee7e"} Sep 08 08:47:55.201076 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:55.201049 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lmwtf" event={"ID":"72a88ec1-2663-4148-9b6f-0b540b3fb77c","Type":"ContainerStarted","Data":"f6f6552eb2d83776dcd1d031d0096f4ebe2d218627fdff7a67d41c4d93fa4274"} Sep 08 08:47:55.202722 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:55.202702 2554 generic.go:356] "Generic (PLEG): container finished" podID="4e49d642-1176-4297-84fd-d0cef9884bff" containerID="5ad1305f21bd61323a9abec3bcfd21d9e58110da4a346468cbf5a0c19957a0ab" exitCode=0 Sep 08 08:47:55.202807 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:55.202773 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wl97n" event={"ID":"4e49d642-1176-4297-84fd-d0cef9884bff","Type":"ContainerDied","Data":"5ad1305f21bd61323a9abec3bcfd21d9e58110da4a346468cbf5a0c19957a0ab"} Sep 08 08:47:55.204767 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:55.204743 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lbvgv" event={"ID":"a4db69aa-e494-477b-8263-f4c7d54f0d71","Type":"ContainerStarted","Data":"8d86a6fd30590c58bf89c2a0a71075aceca8d4875d0a95cf070af23197d4bf2a"} Sep 08 08:47:55.206406 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:55.206378 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkwv8" event={"ID":"095842a7-07af-47c9-a386-6a90456cc5c0","Type":"ContainerStarted","Data":"ad03c1c770724114e015e146d1e5e76500ab8e9f99b0eb4cd0d39cb3d76264cf"} Sep 08 08:47:55.211703 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:55.211660 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-177.ec2.internal" podStartSLOduration=21.211643125 podStartE2EDuration="21.211643125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 08:47:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:47:39.173489069 +0000 UTC m=+5.681595832" watchObservedRunningTime="2026-09-08 08:47:55.211643125 +0000 UTC m=+21.719749894" Sep 08 08:47:55.211867 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:55.211842 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-txfv4" podStartSLOduration=3.307152422 podStartE2EDuration="21.21183292s" totalImagesPullingTime="17.904680498s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 08:47:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:47:55.211344985 +0000 UTC m=+21.719451754" watchObservedRunningTime="2026-09-08 08:47:55.21183292 +0000 UTC m=+21.719939688" Sep 08 08:47:55.228248 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:55.228200 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-lbvgv" podStartSLOduration=3.584372343 podStartE2EDuration="21.228182303s" totalImagesPullingTime="17.64380996s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 08:47:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:47:55.227971588 +0000 UTC m=+21.736078356" watchObservedRunningTime="2026-09-08 08:47:55.228182303 +0000 UTC m=+21.736289072" Sep 08 08:47:55.241802 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:55.241554 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-8bx7f" podStartSLOduration=7.371894058 podStartE2EDuration="21.241543871s" totalImagesPullingTime="13.869649813s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 08:47:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:47:55.241514526 +0000 UTC m=+21.749621293" watchObservedRunningTime="2026-09-08 08:47:55.241543871 +0000 UTC m=+21.749650640" Sep 08 08:47:55.256719 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:55.256678 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-lmwtf" podStartSLOduration=3.617855451 podStartE2EDuration="21.256664912s" totalImagesPullingTime="17.638809461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 08:47:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:47:55.256334137 +0000 UTC m=+21.764440920" watchObservedRunningTime="2026-09-08 08:47:55.256664912 +0000 UTC m=+21.764771680" Sep 08 08:47:55.300726 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:55.300684 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-gxjvt" podStartSLOduration=3.655463942 podStartE2EDuration="21.300669752s" totalImagesPullingTime="17.64520581s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 08:47:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:47:55.276747771 +0000 UTC m=+21.784854561" watchObservedRunningTime="2026-09-08 08:47:55.300669752 +0000 UTC m=+21.808776520" Sep 08 08:47:56.026684 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:56.026649 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6s4m" Sep 08 08:47:56.026815 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:56.026779 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6s4m" podUID="1746be3a-25ff-4dc5-b4fb-6075f46391e0" Sep 08 08:47:56.211960 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:56.211934 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xn2lh_04c4d313-ed7e-4fea-bec5-3ed11857a8c4/ovn-acl-logging/0.log" Sep 08 08:47:56.212742 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:56.212397 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" event={"ID":"04c4d313-ed7e-4fea-bec5-3ed11857a8c4","Type":"ContainerStarted","Data":"cef97b792fdc561b41a33976092e8f37c4c22c4dc781983aa6fbbf0be81099e7"} Sep 08 08:47:56.212742 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:56.212477 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" event={"ID":"04c4d313-ed7e-4fea-bec5-3ed11857a8c4","Type":"ContainerStarted","Data":"28cdc57667e1a7b114b960e762d93bb1a5380ef4c13f451877ad57065ff0c8c1"} Sep 08 08:47:56.212742 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:56.212520 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" event={"ID":"04c4d313-ed7e-4fea-bec5-3ed11857a8c4","Type":"ContainerStarted","Data":"38040cff71dea2f91c3f5101551f50db1fdbf37027e260d517c4ff371bc61e38"} Sep 08 08:47:56.212742 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:56.212561 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" event={"ID":"04c4d313-ed7e-4fea-bec5-3ed11857a8c4","Type":"ContainerStarted","Data":"ac8f2f190429f0aa9819821c752c779dbca18f498d1ed3033902587e7334446f"} Sep 08 08:47:56.214104 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:56.214075 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nsz5v" event={"ID":"ab4bc45b-8015-494b-b088-5f699f96acaa","Type":"ContainerStarted","Data":"563363e87a31f252b3c5011d4fa3a7b1024d20317fe7cd5a7e7ec9fde7c17f30"} Sep 08 08:47:56.229498 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:56.229454 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nsz5v" podStartSLOduration=4.571518001 podStartE2EDuration="22.22943889s" totalImagesPullingTime="17.657920889s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 08:47:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:47:56.229399495 +0000 UTC m=+22.737506267" watchObservedRunningTime="2026-09-08 08:47:56.22943889 +0000 UTC m=+22.737545660" Sep 08 08:47:56.557232 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:56.557209 2554 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 08 08:47:56.949370 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:56.949242 2554 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-08T08:47:56.5572289Z","UUID":"b06e3f40-bbe2-4c54-8898-ef7f7ef24b7e","Handler":null,"Name":"","Endpoint":""} Sep 08 08:47:56.950949 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:56.950919 2554 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 08 08:47:56.951104 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:56.950955 2554 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 08 08:47:57.026896 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:57.026868 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xk7m" Sep 08 08:47:57.027065 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:57.026901 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4smd" Sep 08 08:47:57.027065 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:57.026972 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xk7m" podUID="57716ea8-fa5c-47c2-a186-7a0acbf6ae38" Sep 08 08:47:57.027192 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:57.027098 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4smd" podUID="5e400e6f-9bee-43f4-b6b1-f75ba65c39fc" Sep 08 08:47:57.218958 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:57.218881 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkwv8" event={"ID":"095842a7-07af-47c9-a386-6a90456cc5c0","Type":"ContainerStarted","Data":"fd7becfc86570fdbf2f5df80c44c22c5cf0194692f3ff79e977801b07b3fb52c"} Sep 08 08:47:58.027084 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:58.027055 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6s4m" Sep 08 08:47:58.027186 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:58.027170 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6s4m" podUID="1746be3a-25ff-4dc5-b4fb-6075f46391e0" Sep 08 08:47:58.224219 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:58.224195 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xn2lh_04c4d313-ed7e-4fea-bec5-3ed11857a8c4/ovn-acl-logging/0.log" Sep 08 08:47:58.224582 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:58.224540 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" event={"ID":"04c4d313-ed7e-4fea-bec5-3ed11857a8c4","Type":"ContainerStarted","Data":"14b91c1024c9f59a3851b8984ec4e7f624a934e0220f7a456cba58f3849b7515"} Sep 08 08:47:58.226525 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:58.226500 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkwv8" event={"ID":"095842a7-07af-47c9-a386-6a90456cc5c0","Type":"ContainerStarted","Data":"0022d7f164962e9bdf02763114c343455358d8e0c2e6c19e6b935dd96fd0ff1b"} Sep 08 08:47:58.244889 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:58.244850 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkwv8" podStartSLOduration=3.895862515 podStartE2EDuration="24.244837542s" totalImagesPullingTime="20.348975027s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 08:47:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:47:58.244560134 +0000 UTC m=+24.752666903" watchObservedRunningTime="2026-09-08 08:47:58.244837542 +0000 UTC m=+24.752944310" Sep 08 08:47:58.328045 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:58.328021 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-lmwtf" Sep 08 08:47:58.328523 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:58.328496 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-lmwtf" Sep 08 08:47:59.027418 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:59.027391 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4smd" Sep 08 08:47:59.027418 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:59.027405 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xk7m" Sep 08 08:47:59.027641 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:59.027494 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4smd" podUID="5e400e6f-9bee-43f4-b6b1-f75ba65c39fc" Sep 08 08:47:59.027641 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:47:59.027627 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xk7m" podUID="57716ea8-fa5c-47c2-a186-7a0acbf6ae38" Sep 08 08:47:59.229144 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:59.229085 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-lmwtf" Sep 08 08:47:59.229568 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:47:59.229465 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-lmwtf" Sep 08 08:48:00.026805 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:00.026684 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6s4m" Sep 08 08:48:00.026914 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:00.026880 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6s4m" podUID="1746be3a-25ff-4dc5-b4fb-6075f46391e0" Sep 08 08:48:00.235257 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:00.235191 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xn2lh_04c4d313-ed7e-4fea-bec5-3ed11857a8c4/ovn-acl-logging/0.log" Sep 08 08:48:00.236174 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:00.235632 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" event={"ID":"04c4d313-ed7e-4fea-bec5-3ed11857a8c4","Type":"ContainerStarted","Data":"58caa5e2854260523fd3269779b83f3850037787e86570fc63305be330015071"} Sep 08 08:48:00.236174 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:00.235855 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:48:00.236174 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:00.236065 2554 scope.go:117] "RemoveContainer" containerID="655d77035a31db3f450fa709f437e625009c02cebf7d27bf8f2431d66212bf1b" Sep 08 08:48:00.237711 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:00.237681 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wl97n" event={"ID":"4e49d642-1176-4297-84fd-d0cef9884bff","Type":"ContainerStarted","Data":"ecf7edccb8fea4b5c540f05d2dabb41089d2c6d97f69fafe2fc85ebf1c23e3dc"} Sep 08 08:48:00.251879 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:00.251300 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:48:01.026749 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:01.026721 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4smd" Sep 08 08:48:01.026872 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:01.026728 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xk7m" Sep 08 08:48:01.026872 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:01.026811 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4smd" podUID="5e400e6f-9bee-43f4-b6b1-f75ba65c39fc" Sep 08 08:48:01.026997 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:01.026923 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xk7m" podUID="57716ea8-fa5c-47c2-a186-7a0acbf6ae38" Sep 08 08:48:01.246084 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:01.245995 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xn2lh_04c4d313-ed7e-4fea-bec5-3ed11857a8c4/ovn-acl-logging/0.log" Sep 08 08:48:01.246525 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:01.246493 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" event={"ID":"04c4d313-ed7e-4fea-bec5-3ed11857a8c4","Type":"ContainerStarted","Data":"bc282e2dedf8461e7fe8c92aeae31c86bdc1658bb20c0e2b81148d80d330222c"} Sep 08 08:48:01.247044 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:01.247013 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:48:01.247161 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:01.247104 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:48:01.249234 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:01.249204 2554 generic.go:356] "Generic (PLEG): container finished" podID="4e49d642-1176-4297-84fd-d0cef9884bff" containerID="ecf7edccb8fea4b5c540f05d2dabb41089d2c6d97f69fafe2fc85ebf1c23e3dc" exitCode=0 Sep 08 08:48:01.249342 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:01.249288 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wl97n" event={"ID":"4e49d642-1176-4297-84fd-d0cef9884bff","Type":"ContainerDied","Data":"ecf7edccb8fea4b5c540f05d2dabb41089d2c6d97f69fafe2fc85ebf1c23e3dc"} Sep 08 08:48:01.264808 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:01.264578 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:48:01.308240 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:01.308171 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" podStartSLOduration=9.628646146 podStartE2EDuration="27.308142854s" totalImagesPullingTime="17.679496708s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 08:47:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:48:01.281548997 +0000 UTC m=+27.789655765" watchObservedRunningTime="2026-09-08 08:48:01.308142854 +0000 UTC m=+27.816249623" Sep 08 08:48:02.011861 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:02.011828 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-b6s4m"] Sep 08 08:48:02.012057 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:02.011960 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6s4m" Sep 08 08:48:02.012122 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:02.012086 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6s4m" podUID="1746be3a-25ff-4dc5-b4fb-6075f46391e0" Sep 08 08:48:02.014432 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:02.014408 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z4smd"] Sep 08 08:48:02.014555 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:02.014497 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4smd" Sep 08 08:48:02.014627 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:02.014588 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4smd" podUID="5e400e6f-9bee-43f4-b6b1-f75ba65c39fc" Sep 08 08:48:02.014966 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:02.014940 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9xk7m"] Sep 08 08:48:02.015079 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:02.015041 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xk7m" Sep 08 08:48:02.015139 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:02.015122 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xk7m" podUID="57716ea8-fa5c-47c2-a186-7a0acbf6ae38" Sep 08 08:48:03.254760 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:03.254729 2554 generic.go:356] "Generic (PLEG): container finished" podID="4e49d642-1176-4297-84fd-d0cef9884bff" containerID="45cbe39fedbbd74f1d974abe94946bd07e92ee053a2bdba53460e3201a687d7e" exitCode=0 Sep 08 08:48:03.255195 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:03.254809 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wl97n" event={"ID":"4e49d642-1176-4297-84fd-d0cef9884bff","Type":"ContainerDied","Data":"45cbe39fedbbd74f1d974abe94946bd07e92ee053a2bdba53460e3201a687d7e"} Sep 08 08:48:04.028020 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:04.027995 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4smd" Sep 08 08:48:04.028098 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:04.028032 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xk7m" Sep 08 08:48:04.028131 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:04.028103 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4smd" podUID="5e400e6f-9bee-43f4-b6b1-f75ba65c39fc" Sep 08 08:48:04.028226 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:04.028188 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xk7m" podUID="57716ea8-fa5c-47c2-a186-7a0acbf6ae38" Sep 08 08:48:04.028283 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:04.028269 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6s4m" Sep 08 08:48:04.028365 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:04.028351 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6s4m" podUID="1746be3a-25ff-4dc5-b4fb-6075f46391e0" Sep 08 08:48:04.259674 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:04.259645 2554 generic.go:356] "Generic (PLEG): container finished" podID="4e49d642-1176-4297-84fd-d0cef9884bff" containerID="080d04604a579aae3e2e227bfae441bc0896eb1a691b01a420d5f03c617570de" exitCode=0 Sep 08 08:48:04.260031 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:04.259693 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wl97n" event={"ID":"4e49d642-1176-4297-84fd-d0cef9884bff","Type":"ContainerDied","Data":"080d04604a579aae3e2e227bfae441bc0896eb1a691b01a420d5f03c617570de"} Sep 08 08:48:06.027577 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:06.027543 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6s4m" Sep 08 08:48:06.027968 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:06.027543 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xk7m" Sep 08 08:48:06.027968 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:06.027668 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4smd" Sep 08 08:48:06.027968 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:06.027657 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6s4m" podUID="1746be3a-25ff-4dc5-b4fb-6075f46391e0" Sep 08 08:48:06.027968 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:06.027784 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xk7m" podUID="57716ea8-fa5c-47c2-a186-7a0acbf6ae38" Sep 08 08:48:06.027968 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:06.027865 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4smd" podUID="5e400e6f-9bee-43f4-b6b1-f75ba65c39fc" Sep 08 08:48:07.640663 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:07.640634 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57716ea8-fa5c-47c2-a186-7a0acbf6ae38-metrics-certs\") pod \"network-metrics-daemon-9xk7m\" (UID: \"57716ea8-fa5c-47c2-a186-7a0acbf6ae38\") " pod="openshift-multus/network-metrics-daemon-9xk7m" Sep 08 08:48:07.641233 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:07.640750 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 08:48:07.641233 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:07.640772 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dq6cj\" (UniqueName: \"kubernetes.io/projected/5e400e6f-9bee-43f4-b6b1-f75ba65c39fc-kube-api-access-dq6cj\") pod \"network-check-target-z4smd\" (UID: \"5e400e6f-9bee-43f4-b6b1-f75ba65c39fc\") " pod="openshift-network-diagnostics/network-check-target-z4smd" Sep 08 08:48:07.641233 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:07.640803 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57716ea8-fa5c-47c2-a186-7a0acbf6ae38-metrics-certs podName:57716ea8-fa5c-47c2-a186-7a0acbf6ae38 nodeName:}" failed. No retries permitted until 2026-09-08 08:48:39.640788292 +0000 UTC m=+66.148895038 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57716ea8-fa5c-47c2-a186-7a0acbf6ae38-metrics-certs") pod "network-metrics-daemon-9xk7m" (UID: "57716ea8-fa5c-47c2-a186-7a0acbf6ae38") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 08:48:07.641233 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:07.640876 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1746be3a-25ff-4dc5-b4fb-6075f46391e0-original-pull-secret\") pod \"global-pull-secret-syncer-b6s4m\" (UID: \"1746be3a-25ff-4dc5-b4fb-6075f46391e0\") " pod="kube-system/global-pull-secret-syncer-b6s4m" Sep 08 08:48:07.641233 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:07.640890 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 08:48:07.641233 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:07.640908 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 08:48:07.641233 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:07.640920 2554 projected.go:196] Error preparing data for projected volume kube-api-access-dq6cj for pod openshift-network-diagnostics/network-check-target-z4smd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 08:48:07.641233 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:07.640966 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 08:48:07.641233 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:07.640974 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e400e6f-9bee-43f4-b6b1-f75ba65c39fc-kube-api-access-dq6cj podName:5e400e6f-9bee-43f4-b6b1-f75ba65c39fc nodeName:}" failed. No retries permitted until 2026-09-08 08:48:39.640958665 +0000 UTC m=+66.149065428 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-dq6cj" (UniqueName: "kubernetes.io/projected/5e400e6f-9bee-43f4-b6b1-f75ba65c39fc-kube-api-access-dq6cj") pod "network-check-target-z4smd" (UID: "5e400e6f-9bee-43f4-b6b1-f75ba65c39fc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 08:48:07.641233 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:07.641022 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1746be3a-25ff-4dc5-b4fb-6075f46391e0-original-pull-secret podName:1746be3a-25ff-4dc5-b4fb-6075f46391e0 nodeName:}" failed. No retries permitted until 2026-09-08 08:48:39.641007185 +0000 UTC m=+66.149113930 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1746be3a-25ff-4dc5-b4fb-6075f46391e0-original-pull-secret") pod "global-pull-secret-syncer-b6s4m" (UID: "1746be3a-25ff-4dc5-b4fb-6075f46391e0") : object "kube-system"/"original-pull-secret" not registered Sep 08 08:48:08.026928 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.026730 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4smd" Sep 08 08:48:08.027121 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.026740 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6s4m" Sep 08 08:48:08.027121 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:08.027040 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4smd" podUID="5e400e6f-9bee-43f4-b6b1-f75ba65c39fc" Sep 08 08:48:08.027121 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:08.027093 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b6s4m" podUID="1746be3a-25ff-4dc5-b4fb-6075f46391e0" Sep 08 08:48:08.027121 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.026786 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xk7m" Sep 08 08:48:08.027321 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:08.027197 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9xk7m" podUID="57716ea8-fa5c-47c2-a186-7a0acbf6ae38" Sep 08 08:48:08.733028 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.732933 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-177.ec2.internal" event="NodeReady" Sep 08 08:48:08.733453 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.733086 2554 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 08 08:48:08.765798 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.765772 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-77696969ff-xzvfk"] Sep 08 08:48:08.796932 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.796896 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77696969ff-xzvfk"] Sep 08 08:48:08.796932 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.796936 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-wcgdd"] Sep 08 08:48:08.797166 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.797034 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77696969ff-xzvfk" Sep 08 08:48:08.799726 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.799693 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 08 08:48:08.800171 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.800148 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-kl6tk\"" Sep 08 08:48:08.801018 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.800368 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 08 08:48:08.803932 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.803911 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 08 08:48:08.809236 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.809183 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 08 08:48:08.810694 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.810678 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-lp6dm"] Sep 08 08:48:08.810828 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.810808 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wcgdd" Sep 08 08:48:08.813072 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.813052 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 08 08:48:08.813359 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.813281 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pkgjn\"" Sep 08 08:48:08.813537 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.813519 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 08 08:48:08.829308 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.829289 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lp6dm"] Sep 08 08:48:08.829398 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.829323 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wcgdd"] Sep 08 08:48:08.829447 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.829412 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lp6dm" Sep 08 08:48:08.831776 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.831757 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-hpdn6\"" Sep 08 08:48:08.831776 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.831770 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 08 08:48:08.831911 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.831802 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 08 08:48:08.832176 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.832158 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 08 08:48:08.951214 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.951181 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/47fa42d5-9f58-489c-8031-4a366e32dfc1-config-volume\") pod \"dns-default-wcgdd\" (UID: \"47fa42d5-9f58-489c-8031-4a366e32dfc1\") " pod="openshift-dns/dns-default-wcgdd" Sep 08 08:48:08.951382 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.951221 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-image-registry-private-configuration\") pod \"image-registry-77696969ff-xzvfk\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " pod="openshift-image-registry/image-registry-77696969ff-xzvfk" Sep 08 08:48:08.951382 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.951242 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-ca-trust-extracted\") pod \"image-registry-77696969ff-xzvfk\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " pod="openshift-image-registry/image-registry-77696969ff-xzvfk" Sep 08 08:48:08.951382 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.951323 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/47fa42d5-9f58-489c-8031-4a366e32dfc1-metrics-tls\") pod \"dns-default-wcgdd\" (UID: \"47fa42d5-9f58-489c-8031-4a366e32dfc1\") " pod="openshift-dns/dns-default-wcgdd" Sep 08 08:48:08.951382 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.951356 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/923c000a-89fd-4a8f-a8ff-5aa9a32c3509-cert\") pod \"ingress-canary-lp6dm\" (UID: \"923c000a-89fd-4a8f-a8ff-5aa9a32c3509\") " pod="openshift-ingress-canary/ingress-canary-lp6dm" Sep 08 08:48:08.951557 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.951388 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5278k\" (UniqueName: \"kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-kube-api-access-5278k\") pod \"image-registry-77696969ff-xzvfk\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " pod="openshift-image-registry/image-registry-77696969ff-xzvfk" Sep 08 08:48:08.951557 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.951412 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-installation-pull-secrets\") pod \"image-registry-77696969ff-xzvfk\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " pod="openshift-image-registry/image-registry-77696969ff-xzvfk" Sep 08 08:48:08.951557 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.951458 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmhxt\" (UniqueName: \"kubernetes.io/projected/923c000a-89fd-4a8f-a8ff-5aa9a32c3509-kube-api-access-vmhxt\") pod \"ingress-canary-lp6dm\" (UID: \"923c000a-89fd-4a8f-a8ff-5aa9a32c3509\") " pod="openshift-ingress-canary/ingress-canary-lp6dm" Sep 08 08:48:08.951557 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.951478 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-bound-sa-token\") pod \"image-registry-77696969ff-xzvfk\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " pod="openshift-image-registry/image-registry-77696969ff-xzvfk" Sep 08 08:48:08.951557 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.951495 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47fa42d5-9f58-489c-8031-4a366e32dfc1-tmp-dir\") pod \"dns-default-wcgdd\" (UID: \"47fa42d5-9f58-489c-8031-4a366e32dfc1\") " pod="openshift-dns/dns-default-wcgdd" Sep 08 08:48:08.951557 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.951515 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-trusted-ca\") pod \"image-registry-77696969ff-xzvfk\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " pod="openshift-image-registry/image-registry-77696969ff-xzvfk" Sep 08 08:48:08.951777 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.951563 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-registry-tls\") pod \"image-registry-77696969ff-xzvfk\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " pod="openshift-image-registry/image-registry-77696969ff-xzvfk" Sep 08 08:48:08.951777 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.951587 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-registry-certificates\") pod \"image-registry-77696969ff-xzvfk\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " pod="openshift-image-registry/image-registry-77696969ff-xzvfk" Sep 08 08:48:08.951777 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:08.951611 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9mvj\" (UniqueName: \"kubernetes.io/projected/47fa42d5-9f58-489c-8031-4a366e32dfc1-kube-api-access-k9mvj\") pod \"dns-default-wcgdd\" (UID: \"47fa42d5-9f58-489c-8031-4a366e32dfc1\") " pod="openshift-dns/dns-default-wcgdd" Sep 08 08:48:09.052904 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:09.052823 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/47fa42d5-9f58-489c-8031-4a366e32dfc1-config-volume\") pod \"dns-default-wcgdd\" (UID: \"47fa42d5-9f58-489c-8031-4a366e32dfc1\") " pod="openshift-dns/dns-default-wcgdd" Sep 08 08:48:09.052904 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:09.052869 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-image-registry-private-configuration\") pod \"image-registry-77696969ff-xzvfk\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " pod="openshift-image-registry/image-registry-77696969ff-xzvfk" Sep 08 08:48:09.052904 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:09.052890 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-ca-trust-extracted\") pod \"image-registry-77696969ff-xzvfk\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " pod="openshift-image-registry/image-registry-77696969ff-xzvfk" Sep 08 08:48:09.053190 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:09.053057 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/47fa42d5-9f58-489c-8031-4a366e32dfc1-metrics-tls\") pod \"dns-default-wcgdd\" (UID: \"47fa42d5-9f58-489c-8031-4a366e32dfc1\") " pod="openshift-dns/dns-default-wcgdd" Sep 08 08:48:09.053190 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:09.053094 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/923c000a-89fd-4a8f-a8ff-5aa9a32c3509-cert\") pod \"ingress-canary-lp6dm\" (UID: \"923c000a-89fd-4a8f-a8ff-5aa9a32c3509\") " pod="openshift-ingress-canary/ingress-canary-lp6dm" Sep 08 08:48:09.053190 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:09.053123 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5278k\" (UniqueName: \"kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-kube-api-access-5278k\") pod \"image-registry-77696969ff-xzvfk\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " pod="openshift-image-registry/image-registry-77696969ff-xzvfk" Sep 08 08:48:09.053190 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:09.053159 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-installation-pull-secrets\") pod \"image-registry-77696969ff-xzvfk\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " pod="openshift-image-registry/image-registry-77696969ff-xzvfk" Sep 08 08:48:09.053190 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:09.053174 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 08:48:09.053438 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:09.053205 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmhxt\" (UniqueName: \"kubernetes.io/projected/923c000a-89fd-4a8f-a8ff-5aa9a32c3509-kube-api-access-vmhxt\") pod \"ingress-canary-lp6dm\" (UID: \"923c000a-89fd-4a8f-a8ff-5aa9a32c3509\") " pod="openshift-ingress-canary/ingress-canary-lp6dm" Sep 08 08:48:09.053438 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:09.053243 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47fa42d5-9f58-489c-8031-4a366e32dfc1-metrics-tls podName:47fa42d5-9f58-489c-8031-4a366e32dfc1 nodeName:}" failed. No retries permitted until 2026-09-08 08:48:09.553221991 +0000 UTC m=+36.061328755 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/47fa42d5-9f58-489c-8031-4a366e32dfc1-metrics-tls") pod "dns-default-wcgdd" (UID: "47fa42d5-9f58-489c-8031-4a366e32dfc1") : secret "dns-default-metrics-tls" not found Sep 08 08:48:09.053438 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:09.053267 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-ca-trust-extracted\") pod \"image-registry-77696969ff-xzvfk\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " pod="openshift-image-registry/image-registry-77696969ff-xzvfk" Sep 08 08:48:09.053438 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:09.053268 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-bound-sa-token\") pod \"image-registry-77696969ff-xzvfk\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " pod="openshift-image-registry/image-registry-77696969ff-xzvfk" Sep 08 08:48:09.053438 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:09.053429 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47fa42d5-9f58-489c-8031-4a366e32dfc1-tmp-dir\") pod \"dns-default-wcgdd\" (UID: \"47fa42d5-9f58-489c-8031-4a366e32dfc1\") " pod="openshift-dns/dns-default-wcgdd" Sep 08 08:48:09.053699 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:09.053454 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-trusted-ca\") pod \"image-registry-77696969ff-xzvfk\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " pod="openshift-image-registry/image-registry-77696969ff-xzvfk" Sep 08 08:48:09.053699 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:09.053481 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-registry-tls\") pod \"image-registry-77696969ff-xzvfk\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " pod="openshift-image-registry/image-registry-77696969ff-xzvfk" Sep 08 08:48:09.053699 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:09.053488 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/47fa42d5-9f58-489c-8031-4a366e32dfc1-config-volume\") pod \"dns-default-wcgdd\" (UID: \"47fa42d5-9f58-489c-8031-4a366e32dfc1\") " pod="openshift-dns/dns-default-wcgdd" Sep 08 08:48:09.053699 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:09.053505 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-registry-certificates\") pod \"image-registry-77696969ff-xzvfk\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " pod="openshift-image-registry/image-registry-77696969ff-xzvfk" Sep 08 08:48:09.053699 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:09.053541 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9mvj\" (UniqueName: \"kubernetes.io/projected/47fa42d5-9f58-489c-8031-4a366e32dfc1-kube-api-access-k9mvj\") pod \"dns-default-wcgdd\" (UID: \"47fa42d5-9f58-489c-8031-4a366e32dfc1\") " pod="openshift-dns/dns-default-wcgdd" Sep 08 08:48:09.053699 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:09.053682 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 08 08:48:09.053699 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:09.053698 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77696969ff-xzvfk: secret "image-registry-tls" not found Sep 08 08:48:09.054056 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:09.053741 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-registry-tls podName:c7af2e1e-49c0-4489-9f5b-32bb9d086ccd nodeName:}" failed. No retries permitted until 2026-09-08 08:48:09.55372383 +0000 UTC m=+36.061830576 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-registry-tls") pod "image-registry-77696969ff-xzvfk" (UID: "c7af2e1e-49c0-4489-9f5b-32bb9d086ccd") : secret "image-registry-tls" not found Sep 08 08:48:09.054056 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:09.053968 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47fa42d5-9f58-489c-8031-4a366e32dfc1-tmp-dir\") pod \"dns-default-wcgdd\" (UID: \"47fa42d5-9f58-489c-8031-4a366e32dfc1\") " pod="openshift-dns/dns-default-wcgdd" Sep 08 08:48:09.054278 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:09.054244 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-registry-certificates\") pod \"image-registry-77696969ff-xzvfk\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " pod="openshift-image-registry/image-registry-77696969ff-xzvfk" Sep 08 08:48:09.054444 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:09.054401 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-trusted-ca\") pod \"image-registry-77696969ff-xzvfk\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " pod="openshift-image-registry/image-registry-77696969ff-xzvfk" Sep 08 08:48:09.057493 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:09.057475 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-image-registry-private-configuration\") pod \"image-registry-77696969ff-xzvfk\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " pod="openshift-image-registry/image-registry-77696969ff-xzvfk" Sep 08 08:48:09.057604 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:09.057525 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-installation-pull-secrets\") pod \"image-registry-77696969ff-xzvfk\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " pod="openshift-image-registry/image-registry-77696969ff-xzvfk" Sep 08 08:48:09.057604 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:09.057564 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/923c000a-89fd-4a8f-a8ff-5aa9a32c3509-cert\") pod \"ingress-canary-lp6dm\" (UID: \"923c000a-89fd-4a8f-a8ff-5aa9a32c3509\") " pod="openshift-ingress-canary/ingress-canary-lp6dm" Sep 08 08:48:09.063330 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:09.063306 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-bound-sa-token\") pod \"image-registry-77696969ff-xzvfk\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " pod="openshift-image-registry/image-registry-77696969ff-xzvfk" Sep 08 08:48:09.063448 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:09.063428 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5278k\" (UniqueName: \"kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-kube-api-access-5278k\") pod \"image-registry-77696969ff-xzvfk\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " pod="openshift-image-registry/image-registry-77696969ff-xzvfk" Sep 08 08:48:09.063489 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:09.063468 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmhxt\" (UniqueName: \"kubernetes.io/projected/923c000a-89fd-4a8f-a8ff-5aa9a32c3509-kube-api-access-vmhxt\") pod \"ingress-canary-lp6dm\" (UID: \"923c000a-89fd-4a8f-a8ff-5aa9a32c3509\") " pod="openshift-ingress-canary/ingress-canary-lp6dm" Sep 08 08:48:09.063776 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:09.063753 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9mvj\" (UniqueName: \"kubernetes.io/projected/47fa42d5-9f58-489c-8031-4a366e32dfc1-kube-api-access-k9mvj\") pod \"dns-default-wcgdd\" (UID: \"47fa42d5-9f58-489c-8031-4a366e32dfc1\") " pod="openshift-dns/dns-default-wcgdd" Sep 08 08:48:09.137870 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:09.137836 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lp6dm" Sep 08 08:48:09.556299 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:09.556268 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/47fa42d5-9f58-489c-8031-4a366e32dfc1-metrics-tls\") pod \"dns-default-wcgdd\" (UID: \"47fa42d5-9f58-489c-8031-4a366e32dfc1\") " pod="openshift-dns/dns-default-wcgdd" Sep 08 08:48:09.556433 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:09.556316 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-registry-tls\") pod \"image-registry-77696969ff-xzvfk\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " pod="openshift-image-registry/image-registry-77696969ff-xzvfk" Sep 08 08:48:09.556433 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:09.556404 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 08 08:48:09.556433 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:09.556406 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 08:48:09.556538 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:09.556471 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47fa42d5-9f58-489c-8031-4a366e32dfc1-metrics-tls podName:47fa42d5-9f58-489c-8031-4a366e32dfc1 nodeName:}" failed. No retries permitted until 2026-09-08 08:48:10.556453597 +0000 UTC m=+37.064560344 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/47fa42d5-9f58-489c-8031-4a366e32dfc1-metrics-tls") pod "dns-default-wcgdd" (UID: "47fa42d5-9f58-489c-8031-4a366e32dfc1") : secret "dns-default-metrics-tls" not found Sep 08 08:48:09.556538 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:09.556415 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77696969ff-xzvfk: secret "image-registry-tls" not found Sep 08 08:48:09.556661 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:09.556539 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-registry-tls podName:c7af2e1e-49c0-4489-9f5b-32bb9d086ccd nodeName:}" failed. No retries permitted until 2026-09-08 08:48:10.556528393 +0000 UTC m=+37.064635154 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-registry-tls") pod "image-registry-77696969ff-xzvfk" (UID: "c7af2e1e-49c0-4489-9f5b-32bb9d086ccd") : secret "image-registry-tls" not found Sep 08 08:48:09.562412 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:09.562390 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lp6dm"] Sep 08 08:48:09.565383 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:48:09.565354 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod923c000a_89fd_4a8f_a8ff_5aa9a32c3509.slice/crio-95f110cb6437776c48d5e013508a7e5fc731b516a9f5055d843a9ce367d93840 WatchSource:0}: Error finding container 95f110cb6437776c48d5e013508a7e5fc731b516a9f5055d843a9ce367d93840: Status 404 returned error can't find the container with id 95f110cb6437776c48d5e013508a7e5fc731b516a9f5055d843a9ce367d93840 Sep 08 08:48:10.027143 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:10.027118 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xk7m" Sep 08 08:48:10.027784 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:10.027120 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6s4m" Sep 08 08:48:10.027784 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:10.027118 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4smd" Sep 08 08:48:10.030573 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:10.030542 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 08 08:48:10.030706 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:10.030611 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 08 08:48:10.030706 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:10.030557 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dt6vp\"" Sep 08 08:48:10.030706 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:10.030686 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 08 08:48:10.030864 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:10.030574 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-d8k6z\"" Sep 08 08:48:10.031009 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:10.030994 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 08 08:48:10.277226 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:10.277121 2554 generic.go:356] "Generic (PLEG): container finished" podID="4e49d642-1176-4297-84fd-d0cef9884bff" containerID="8050da4d696f9b87260370872c7f62ae2bd6cd5bf448c9936b4badd56de5f0a8" exitCode=0 Sep 08 08:48:10.277226 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:10.277151 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wl97n" event={"ID":"4e49d642-1176-4297-84fd-d0cef9884bff","Type":"ContainerDied","Data":"8050da4d696f9b87260370872c7f62ae2bd6cd5bf448c9936b4badd56de5f0a8"} Sep 08 08:48:10.278920 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:10.278887 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lp6dm" event={"ID":"923c000a-89fd-4a8f-a8ff-5aa9a32c3509","Type":"ContainerStarted","Data":"95f110cb6437776c48d5e013508a7e5fc731b516a9f5055d843a9ce367d93840"} Sep 08 08:48:10.562564 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:10.562483 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/47fa42d5-9f58-489c-8031-4a366e32dfc1-metrics-tls\") pod \"dns-default-wcgdd\" (UID: \"47fa42d5-9f58-489c-8031-4a366e32dfc1\") " pod="openshift-dns/dns-default-wcgdd" Sep 08 08:48:10.562564 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:10.562541 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-registry-tls\") pod \"image-registry-77696969ff-xzvfk\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " pod="openshift-image-registry/image-registry-77696969ff-xzvfk" Sep 08 08:48:10.562782 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:10.562650 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 08 08:48:10.562782 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:10.562653 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 08:48:10.562782 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:10.562734 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47fa42d5-9f58-489c-8031-4a366e32dfc1-metrics-tls podName:47fa42d5-9f58-489c-8031-4a366e32dfc1 nodeName:}" failed. No retries permitted until 2026-09-08 08:48:12.562711498 +0000 UTC m=+39.070818252 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/47fa42d5-9f58-489c-8031-4a366e32dfc1-metrics-tls") pod "dns-default-wcgdd" (UID: "47fa42d5-9f58-489c-8031-4a366e32dfc1") : secret "dns-default-metrics-tls" not found Sep 08 08:48:10.562782 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:10.562665 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77696969ff-xzvfk: secret "image-registry-tls" not found Sep 08 08:48:10.562782 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:10.562786 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-registry-tls podName:c7af2e1e-49c0-4489-9f5b-32bb9d086ccd nodeName:}" failed. No retries permitted until 2026-09-08 08:48:12.562774666 +0000 UTC m=+39.070881412 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-registry-tls") pod "image-registry-77696969ff-xzvfk" (UID: "c7af2e1e-49c0-4489-9f5b-32bb9d086ccd") : secret "image-registry-tls" not found Sep 08 08:48:11.284200 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:11.284170 2554 generic.go:356] "Generic (PLEG): container finished" podID="4e49d642-1176-4297-84fd-d0cef9884bff" containerID="2c9a15e60661fbf45a0269a061dd883b59106a59c4723d30935728efd9a4adaf" exitCode=0 Sep 08 08:48:11.284520 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:11.284234 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wl97n" event={"ID":"4e49d642-1176-4297-84fd-d0cef9884bff","Type":"ContainerDied","Data":"2c9a15e60661fbf45a0269a061dd883b59106a59c4723d30935728efd9a4adaf"} Sep 08 08:48:12.289992 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:12.289943 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wl97n" event={"ID":"4e49d642-1176-4297-84fd-d0cef9884bff","Type":"ContainerStarted","Data":"a171bfc8c4484117ddc7f78f73b72f7199ca111c5a6435414c06ec0e4e186046"} Sep 08 08:48:12.311826 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:12.311754 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wl97n" podStartSLOduration=8.729550338 podStartE2EDuration="38.311738983s" totalImagesPullingTime="29.582188645s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 08:47:34 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:48:12.310051065 +0000 UTC m=+38.818157832" watchObservedRunningTime="2026-09-08 08:48:12.311738983 +0000 UTC m=+38.819845747" Sep 08 08:48:12.574853 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:12.574774 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/47fa42d5-9f58-489c-8031-4a366e32dfc1-metrics-tls\") pod \"dns-default-wcgdd\" (UID: \"47fa42d5-9f58-489c-8031-4a366e32dfc1\") " pod="openshift-dns/dns-default-wcgdd" Sep 08 08:48:12.574853 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:12.574826 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-registry-tls\") pod \"image-registry-77696969ff-xzvfk\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " pod="openshift-image-registry/image-registry-77696969ff-xzvfk" Sep 08 08:48:12.575046 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:12.574926 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 08:48:12.575046 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:12.574940 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 08 08:48:12.575046 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:12.574950 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77696969ff-xzvfk: secret "image-registry-tls" not found Sep 08 08:48:12.575046 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:12.575009 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47fa42d5-9f58-489c-8031-4a366e32dfc1-metrics-tls podName:47fa42d5-9f58-489c-8031-4a366e32dfc1 nodeName:}" failed. No retries permitted until 2026-09-08 08:48:16.574989687 +0000 UTC m=+43.083096446 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/47fa42d5-9f58-489c-8031-4a366e32dfc1-metrics-tls") pod "dns-default-wcgdd" (UID: "47fa42d5-9f58-489c-8031-4a366e32dfc1") : secret "dns-default-metrics-tls" not found Sep 08 08:48:12.575046 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:12.575024 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-registry-tls podName:c7af2e1e-49c0-4489-9f5b-32bb9d086ccd nodeName:}" failed. No retries permitted until 2026-09-08 08:48:16.575017931 +0000 UTC m=+43.083124681 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-registry-tls") pod "image-registry-77696969ff-xzvfk" (UID: "c7af2e1e-49c0-4489-9f5b-32bb9d086ccd") : secret "image-registry-tls" not found Sep 08 08:48:13.293917 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:13.293881 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lp6dm" event={"ID":"923c000a-89fd-4a8f-a8ff-5aa9a32c3509","Type":"ContainerStarted","Data":"3031ca800403c1266b8a7044e1f08045104abdf86c3c0f31135395af3375f068"} Sep 08 08:48:13.311428 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:13.311383 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-lp6dm" podStartSLOduration=2.5917056929999998 podStartE2EDuration="5.311369729s" totalImagesPullingTime="2.719664036s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 08:48:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:48:13.310669758 +0000 UTC m=+39.818776537" watchObservedRunningTime="2026-09-08 08:48:13.311369729 +0000 UTC m=+39.819476494" Sep 08 08:48:16.595829 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:16.595798 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-registry-tls\") pod \"image-registry-77696969ff-xzvfk\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " pod="openshift-image-registry/image-registry-77696969ff-xzvfk" Sep 08 08:48:16.596198 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:16.595858 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/47fa42d5-9f58-489c-8031-4a366e32dfc1-metrics-tls\") pod \"dns-default-wcgdd\" (UID: \"47fa42d5-9f58-489c-8031-4a366e32dfc1\") " pod="openshift-dns/dns-default-wcgdd" Sep 08 08:48:16.596198 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:16.595940 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 08:48:16.596198 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:16.595999 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47fa42d5-9f58-489c-8031-4a366e32dfc1-metrics-tls podName:47fa42d5-9f58-489c-8031-4a366e32dfc1 nodeName:}" failed. No retries permitted until 2026-09-08 08:48:24.595974088 +0000 UTC m=+51.104080834 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/47fa42d5-9f58-489c-8031-4a366e32dfc1-metrics-tls") pod "dns-default-wcgdd" (UID: "47fa42d5-9f58-489c-8031-4a366e32dfc1") : secret "dns-default-metrics-tls" not found Sep 08 08:48:16.596198 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:16.595939 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 08 08:48:16.596198 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:16.596039 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77696969ff-xzvfk: secret "image-registry-tls" not found Sep 08 08:48:16.596198 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:16.596082 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-registry-tls podName:c7af2e1e-49c0-4489-9f5b-32bb9d086ccd nodeName:}" failed. No retries permitted until 2026-09-08 08:48:24.596071149 +0000 UTC m=+51.104177895 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-registry-tls") pod "image-registry-77696969ff-xzvfk" (UID: "c7af2e1e-49c0-4489-9f5b-32bb9d086ccd") : secret "image-registry-tls" not found Sep 08 08:48:24.650023 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:24.649974 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-registry-tls\") pod \"image-registry-77696969ff-xzvfk\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " pod="openshift-image-registry/image-registry-77696969ff-xzvfk" Sep 08 08:48:24.650527 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:24.650067 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/47fa42d5-9f58-489c-8031-4a366e32dfc1-metrics-tls\") pod \"dns-default-wcgdd\" (UID: \"47fa42d5-9f58-489c-8031-4a366e32dfc1\") " pod="openshift-dns/dns-default-wcgdd" Sep 08 08:48:24.650527 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:24.650130 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 08 08:48:24.650527 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:24.650151 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77696969ff-xzvfk: secret "image-registry-tls" not found Sep 08 08:48:24.650527 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:24.650162 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 08:48:24.650527 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:24.650207 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-registry-tls podName:c7af2e1e-49c0-4489-9f5b-32bb9d086ccd nodeName:}" failed. No retries permitted until 2026-09-08 08:48:40.650191529 +0000 UTC m=+67.158298279 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-registry-tls") pod "image-registry-77696969ff-xzvfk" (UID: "c7af2e1e-49c0-4489-9f5b-32bb9d086ccd") : secret "image-registry-tls" not found Sep 08 08:48:24.650527 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:24.650221 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47fa42d5-9f58-489c-8031-4a366e32dfc1-metrics-tls podName:47fa42d5-9f58-489c-8031-4a366e32dfc1 nodeName:}" failed. No retries permitted until 2026-09-08 08:48:40.650214902 +0000 UTC m=+67.158321648 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/47fa42d5-9f58-489c-8031-4a366e32dfc1-metrics-tls") pod "dns-default-wcgdd" (UID: "47fa42d5-9f58-489c-8031-4a366e32dfc1") : secret "dns-default-metrics-tls" not found Sep 08 08:48:25.596059 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.596028 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-v5vkr"] Sep 08 08:48:25.597765 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.597749 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-v5vkr" Sep 08 08:48:25.600022 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.599996 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Sep 08 08:48:25.600184 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.600169 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Sep 08 08:48:25.600966 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.600946 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-df7ks\"" Sep 08 08:48:25.601065 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.600965 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Sep 08 08:48:25.601065 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.601004 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Sep 08 08:48:25.611616 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.611594 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-v5vkr"] Sep 08 08:48:25.657734 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.657710 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5ee267e6-6baf-499a-9e98-78e38f849123-serving-cert\") pod \"service-ca-operator-859f94c4b-v5vkr\" (UID: \"5ee267e6-6baf-499a-9e98-78e38f849123\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-v5vkr" Sep 08 08:48:25.658065 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.657736 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5ee267e6-6baf-499a-9e98-78e38f849123-config\") pod \"service-ca-operator-859f94c4b-v5vkr\" (UID: \"5ee267e6-6baf-499a-9e98-78e38f849123\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-v5vkr" Sep 08 08:48:25.658065 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.657753 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxgzg\" (UniqueName: \"kubernetes.io/projected/5ee267e6-6baf-499a-9e98-78e38f849123-kube-api-access-gxgzg\") pod \"service-ca-operator-859f94c4b-v5vkr\" (UID: \"5ee267e6-6baf-499a-9e98-78e38f849123\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-v5vkr" Sep 08 08:48:25.705634 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.705589 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hs6rq"] Sep 08 08:48:25.707359 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.707343 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hs6rq" Sep 08 08:48:25.710753 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.710736 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Sep 08 08:48:25.714603 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.714584 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Sep 08 08:48:25.714823 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.714806 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Sep 08 08:48:25.714941 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.714921 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-4zq87"] Sep 08 08:48:25.715959 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.715943 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Sep 08 08:48:25.716064 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.715957 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-trkfr\"" Sep 08 08:48:25.716601 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.716587 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-4zq87" Sep 08 08:48:25.721027 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.721011 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-99dgp\"" Sep 08 08:48:25.734130 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.734110 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hs6rq"] Sep 08 08:48:25.744877 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.744858 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-4zq87"] Sep 08 08:48:25.758151 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.758124 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bd212df8-59bb-46ae-8a18-3d1eb3cbf305-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-hs6rq\" (UID: \"bd212df8-59bb-46ae-8a18-3d1eb3cbf305\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hs6rq" Sep 08 08:48:25.758270 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.758162 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jj6km\" (UniqueName: \"kubernetes.io/projected/b81d4647-152e-4310-8b25-4624a76694e8-kube-api-access-jj6km\") pod \"network-check-source-685456655f-4zq87\" (UID: \"b81d4647-152e-4310-8b25-4624a76694e8\") " pod="openshift-network-diagnostics/network-check-source-685456655f-4zq87" Sep 08 08:48:25.758270 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.758214 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5ee267e6-6baf-499a-9e98-78e38f849123-serving-cert\") pod \"service-ca-operator-859f94c4b-v5vkr\" (UID: \"5ee267e6-6baf-499a-9e98-78e38f849123\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-v5vkr" Sep 08 08:48:25.758270 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.758236 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5ee267e6-6baf-499a-9e98-78e38f849123-config\") pod \"service-ca-operator-859f94c4b-v5vkr\" (UID: \"5ee267e6-6baf-499a-9e98-78e38f849123\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-v5vkr" Sep 08 08:48:25.758270 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.758252 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxgzg\" (UniqueName: \"kubernetes.io/projected/5ee267e6-6baf-499a-9e98-78e38f849123-kube-api-access-gxgzg\") pod \"service-ca-operator-859f94c4b-v5vkr\" (UID: \"5ee267e6-6baf-499a-9e98-78e38f849123\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-v5vkr" Sep 08 08:48:25.758484 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.758297 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bd212df8-59bb-46ae-8a18-3d1eb3cbf305-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-hs6rq\" (UID: \"bd212df8-59bb-46ae-8a18-3d1eb3cbf305\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hs6rq" Sep 08 08:48:25.758484 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.758331 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvn9h\" (UniqueName: \"kubernetes.io/projected/bd212df8-59bb-46ae-8a18-3d1eb3cbf305-kube-api-access-tvn9h\") pod \"kube-storage-version-migrator-operator-8d7dd4675-hs6rq\" (UID: \"bd212df8-59bb-46ae-8a18-3d1eb3cbf305\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hs6rq" Sep 08 08:48:25.758894 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.758875 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5ee267e6-6baf-499a-9e98-78e38f849123-config\") pod \"service-ca-operator-859f94c4b-v5vkr\" (UID: \"5ee267e6-6baf-499a-9e98-78e38f849123\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-v5vkr" Sep 08 08:48:25.761524 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.761506 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5ee267e6-6baf-499a-9e98-78e38f849123-serving-cert\") pod \"service-ca-operator-859f94c4b-v5vkr\" (UID: \"5ee267e6-6baf-499a-9e98-78e38f849123\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-v5vkr" Sep 08 08:48:25.773593 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.773574 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxgzg\" (UniqueName: \"kubernetes.io/projected/5ee267e6-6baf-499a-9e98-78e38f849123-kube-api-access-gxgzg\") pod \"service-ca-operator-859f94c4b-v5vkr\" (UID: \"5ee267e6-6baf-499a-9e98-78e38f849123\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-v5vkr" Sep 08 08:48:25.802703 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.802668 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-xnd5g"] Sep 08 08:48:25.804968 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.804945 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-xnd5g" Sep 08 08:48:25.807607 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.807583 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Sep 08 08:48:25.807692 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.807587 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-8grq5\"" Sep 08 08:48:25.807919 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.807906 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Sep 08 08:48:25.807969 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.807920 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Sep 08 08:48:25.808086 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.808073 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Sep 08 08:48:25.812379 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.812363 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Sep 08 08:48:25.817285 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.817265 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-xnd5g"] Sep 08 08:48:25.859161 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.859100 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/df2da370-7d8d-414d-b82f-3d857fadd070-trusted-ca\") pod \"console-operator-85b454874b-xnd5g\" (UID: \"df2da370-7d8d-414d-b82f-3d857fadd070\") " pod="openshift-console-operator/console-operator-85b454874b-xnd5g" Sep 08 08:48:25.859161 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.859146 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bd212df8-59bb-46ae-8a18-3d1eb3cbf305-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-hs6rq\" (UID: \"bd212df8-59bb-46ae-8a18-3d1eb3cbf305\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hs6rq" Sep 08 08:48:25.859303 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.859176 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jj6km\" (UniqueName: \"kubernetes.io/projected/b81d4647-152e-4310-8b25-4624a76694e8-kube-api-access-jj6km\") pod \"network-check-source-685456655f-4zq87\" (UID: \"b81d4647-152e-4310-8b25-4624a76694e8\") " pod="openshift-network-diagnostics/network-check-source-685456655f-4zq87" Sep 08 08:48:25.859303 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.859268 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/df2da370-7d8d-414d-b82f-3d857fadd070-config\") pod \"console-operator-85b454874b-xnd5g\" (UID: \"df2da370-7d8d-414d-b82f-3d857fadd070\") " pod="openshift-console-operator/console-operator-85b454874b-xnd5g" Sep 08 08:48:25.859303 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.859299 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bd212df8-59bb-46ae-8a18-3d1eb3cbf305-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-hs6rq\" (UID: \"bd212df8-59bb-46ae-8a18-3d1eb3cbf305\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hs6rq" Sep 08 08:48:25.859511 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.859324 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zcgv\" (UniqueName: \"kubernetes.io/projected/df2da370-7d8d-414d-b82f-3d857fadd070-kube-api-access-6zcgv\") pod \"console-operator-85b454874b-xnd5g\" (UID: \"df2da370-7d8d-414d-b82f-3d857fadd070\") " pod="openshift-console-operator/console-operator-85b454874b-xnd5g" Sep 08 08:48:25.859511 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.859371 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvn9h\" (UniqueName: \"kubernetes.io/projected/bd212df8-59bb-46ae-8a18-3d1eb3cbf305-kube-api-access-tvn9h\") pod \"kube-storage-version-migrator-operator-8d7dd4675-hs6rq\" (UID: \"bd212df8-59bb-46ae-8a18-3d1eb3cbf305\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hs6rq" Sep 08 08:48:25.859511 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.859394 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/df2da370-7d8d-414d-b82f-3d857fadd070-serving-cert\") pod \"console-operator-85b454874b-xnd5g\" (UID: \"df2da370-7d8d-414d-b82f-3d857fadd070\") " pod="openshift-console-operator/console-operator-85b454874b-xnd5g" Sep 08 08:48:25.859660 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.859602 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bd212df8-59bb-46ae-8a18-3d1eb3cbf305-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-hs6rq\" (UID: \"bd212df8-59bb-46ae-8a18-3d1eb3cbf305\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hs6rq" Sep 08 08:48:25.861492 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.861474 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bd212df8-59bb-46ae-8a18-3d1eb3cbf305-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-hs6rq\" (UID: \"bd212df8-59bb-46ae-8a18-3d1eb3cbf305\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hs6rq" Sep 08 08:48:25.867239 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.867216 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvn9h\" (UniqueName: \"kubernetes.io/projected/bd212df8-59bb-46ae-8a18-3d1eb3cbf305-kube-api-access-tvn9h\") pod \"kube-storage-version-migrator-operator-8d7dd4675-hs6rq\" (UID: \"bd212df8-59bb-46ae-8a18-3d1eb3cbf305\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hs6rq" Sep 08 08:48:25.867560 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.867541 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jj6km\" (UniqueName: \"kubernetes.io/projected/b81d4647-152e-4310-8b25-4624a76694e8-kube-api-access-jj6km\") pod \"network-check-source-685456655f-4zq87\" (UID: \"b81d4647-152e-4310-8b25-4624a76694e8\") " pod="openshift-network-diagnostics/network-check-source-685456655f-4zq87" Sep 08 08:48:25.892957 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.892933 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-kcwgm"] Sep 08 08:48:25.894705 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.894691 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-kcwgm" Sep 08 08:48:25.897344 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.897324 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Sep 08 08:48:25.897461 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.897328 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Sep 08 08:48:25.897461 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.897328 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-7775r\"" Sep 08 08:48:25.906457 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.906439 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-v5vkr" Sep 08 08:48:25.907450 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.907431 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-kcwgm"] Sep 08 08:48:25.960469 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.960442 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/df2da370-7d8d-414d-b82f-3d857fadd070-config\") pod \"console-operator-85b454874b-xnd5g\" (UID: \"df2da370-7d8d-414d-b82f-3d857fadd070\") " pod="openshift-console-operator/console-operator-85b454874b-xnd5g" Sep 08 08:48:25.960595 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.960476 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zcgv\" (UniqueName: \"kubernetes.io/projected/df2da370-7d8d-414d-b82f-3d857fadd070-kube-api-access-6zcgv\") pod \"console-operator-85b454874b-xnd5g\" (UID: \"df2da370-7d8d-414d-b82f-3d857fadd070\") " pod="openshift-console-operator/console-operator-85b454874b-xnd5g" Sep 08 08:48:25.960595 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.960517 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/df2da370-7d8d-414d-b82f-3d857fadd070-serving-cert\") pod \"console-operator-85b454874b-xnd5g\" (UID: \"df2da370-7d8d-414d-b82f-3d857fadd070\") " pod="openshift-console-operator/console-operator-85b454874b-xnd5g" Sep 08 08:48:25.960595 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.960559 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/df2da370-7d8d-414d-b82f-3d857fadd070-trusted-ca\") pod \"console-operator-85b454874b-xnd5g\" (UID: \"df2da370-7d8d-414d-b82f-3d857fadd070\") " pod="openshift-console-operator/console-operator-85b454874b-xnd5g" Sep 08 08:48:25.960728 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.960605 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/242ea579-b683-4fe7-a860-89724bce185b-nginx-conf\") pod \"networking-console-plugin-79977f9844-kcwgm\" (UID: \"242ea579-b683-4fe7-a860-89724bce185b\") " pod="openshift-network-console/networking-console-plugin-79977f9844-kcwgm" Sep 08 08:48:25.960728 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.960664 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/242ea579-b683-4fe7-a860-89724bce185b-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-kcwgm\" (UID: \"242ea579-b683-4fe7-a860-89724bce185b\") " pod="openshift-network-console/networking-console-plugin-79977f9844-kcwgm" Sep 08 08:48:25.961241 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.961211 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/df2da370-7d8d-414d-b82f-3d857fadd070-config\") pod \"console-operator-85b454874b-xnd5g\" (UID: \"df2da370-7d8d-414d-b82f-3d857fadd070\") " pod="openshift-console-operator/console-operator-85b454874b-xnd5g" Sep 08 08:48:25.962266 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.962238 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/df2da370-7d8d-414d-b82f-3d857fadd070-trusted-ca\") pod \"console-operator-85b454874b-xnd5g\" (UID: \"df2da370-7d8d-414d-b82f-3d857fadd070\") " pod="openshift-console-operator/console-operator-85b454874b-xnd5g" Sep 08 08:48:25.963386 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.963363 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/df2da370-7d8d-414d-b82f-3d857fadd070-serving-cert\") pod \"console-operator-85b454874b-xnd5g\" (UID: \"df2da370-7d8d-414d-b82f-3d857fadd070\") " pod="openshift-console-operator/console-operator-85b454874b-xnd5g" Sep 08 08:48:25.972756 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:25.972737 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zcgv\" (UniqueName: \"kubernetes.io/projected/df2da370-7d8d-414d-b82f-3d857fadd070-kube-api-access-6zcgv\") pod \"console-operator-85b454874b-xnd5g\" (UID: \"df2da370-7d8d-414d-b82f-3d857fadd070\") " pod="openshift-console-operator/console-operator-85b454874b-xnd5g" Sep 08 08:48:26.015514 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:26.015487 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hs6rq" Sep 08 08:48:26.024090 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:26.023963 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-4zq87" Sep 08 08:48:26.064816 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:26.063126 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/242ea579-b683-4fe7-a860-89724bce185b-nginx-conf\") pod \"networking-console-plugin-79977f9844-kcwgm\" (UID: \"242ea579-b683-4fe7-a860-89724bce185b\") " pod="openshift-network-console/networking-console-plugin-79977f9844-kcwgm" Sep 08 08:48:26.064816 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:26.063221 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/242ea579-b683-4fe7-a860-89724bce185b-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-kcwgm\" (UID: \"242ea579-b683-4fe7-a860-89724bce185b\") " pod="openshift-network-console/networking-console-plugin-79977f9844-kcwgm" Sep 08 08:48:26.064816 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:26.063400 2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 08 08:48:26.064816 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:26.063465 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/242ea579-b683-4fe7-a860-89724bce185b-networking-console-plugin-cert podName:242ea579-b683-4fe7-a860-89724bce185b nodeName:}" failed. No retries permitted until 2026-09-08 08:48:26.563446351 +0000 UTC m=+53.071553115 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/242ea579-b683-4fe7-a860-89724bce185b-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-kcwgm" (UID: "242ea579-b683-4fe7-a860-89724bce185b") : secret "networking-console-plugin-cert" not found Sep 08 08:48:26.064816 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:26.064561 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/242ea579-b683-4fe7-a860-89724bce185b-nginx-conf\") pod \"networking-console-plugin-79977f9844-kcwgm\" (UID: \"242ea579-b683-4fe7-a860-89724bce185b\") " pod="openshift-network-console/networking-console-plugin-79977f9844-kcwgm" Sep 08 08:48:26.113957 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:26.113904 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-xnd5g" Sep 08 08:48:26.176750 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:26.176665 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-v5vkr"] Sep 08 08:48:26.179848 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:48:26.179823 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ee267e6_6baf_499a_9e98_78e38f849123.slice/crio-732d5075fa21de52c24dfc8c70b0d8604ceb9054577423323ad49205e101eb48 WatchSource:0}: Error finding container 732d5075fa21de52c24dfc8c70b0d8604ceb9054577423323ad49205e101eb48: Status 404 returned error can't find the container with id 732d5075fa21de52c24dfc8c70b0d8604ceb9054577423323ad49205e101eb48 Sep 08 08:48:26.282646 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:26.282621 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hs6rq"] Sep 08 08:48:26.286486 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:48:26.285947 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd212df8_59bb_46ae_8a18_3d1eb3cbf305.slice/crio-3de08541387465c398e98f0d85878108f3e6402bd276ef160f42de64147e1090 WatchSource:0}: Error finding container 3de08541387465c398e98f0d85878108f3e6402bd276ef160f42de64147e1090: Status 404 returned error can't find the container with id 3de08541387465c398e98f0d85878108f3e6402bd276ef160f42de64147e1090 Sep 08 08:48:26.300592 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:26.300573 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-4zq87"] Sep 08 08:48:26.302951 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:48:26.302926 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb81d4647_152e_4310_8b25_4624a76694e8.slice/crio-ae052acb23ec4bea581c5403b285585b5912742a3d3d0bbc5d6b81be2d6941f4 WatchSource:0}: Error finding container ae052acb23ec4bea581c5403b285585b5912742a3d3d0bbc5d6b81be2d6941f4: Status 404 returned error can't find the container with id ae052acb23ec4bea581c5403b285585b5912742a3d3d0bbc5d6b81be2d6941f4 Sep 08 08:48:26.323777 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:26.323752 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-v5vkr" event={"ID":"5ee267e6-6baf-499a-9e98-78e38f849123","Type":"ContainerStarted","Data":"732d5075fa21de52c24dfc8c70b0d8604ceb9054577423323ad49205e101eb48"} Sep 08 08:48:26.324794 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:26.324771 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hs6rq" event={"ID":"bd212df8-59bb-46ae-8a18-3d1eb3cbf305","Type":"ContainerStarted","Data":"3de08541387465c398e98f0d85878108f3e6402bd276ef160f42de64147e1090"} Sep 08 08:48:26.325747 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:26.325725 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-4zq87" event={"ID":"b81d4647-152e-4310-8b25-4624a76694e8","Type":"ContainerStarted","Data":"ae052acb23ec4bea581c5403b285585b5912742a3d3d0bbc5d6b81be2d6941f4"} Sep 08 08:48:26.380935 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:26.380889 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-xnd5g"] Sep 08 08:48:26.383354 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:48:26.383333 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf2da370_7d8d_414d_b82f_3d857fadd070.slice/crio-46acaaf7674f5f188c31b22e4445eba3454c4b745e911f90c3c9078ef361b1b7 WatchSource:0}: Error finding container 46acaaf7674f5f188c31b22e4445eba3454c4b745e911f90c3c9078ef361b1b7: Status 404 returned error can't find the container with id 46acaaf7674f5f188c31b22e4445eba3454c4b745e911f90c3c9078ef361b1b7 Sep 08 08:48:26.568563 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:26.568526 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/242ea579-b683-4fe7-a860-89724bce185b-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-kcwgm\" (UID: \"242ea579-b683-4fe7-a860-89724bce185b\") " pod="openshift-network-console/networking-console-plugin-79977f9844-kcwgm" Sep 08 08:48:26.568718 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:26.568635 2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 08 08:48:26.568718 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:26.568685 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/242ea579-b683-4fe7-a860-89724bce185b-networking-console-plugin-cert podName:242ea579-b683-4fe7-a860-89724bce185b nodeName:}" failed. No retries permitted until 2026-09-08 08:48:27.568672363 +0000 UTC m=+54.076779108 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/242ea579-b683-4fe7-a860-89724bce185b-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-kcwgm" (UID: "242ea579-b683-4fe7-a860-89724bce185b") : secret "networking-console-plugin-cert" not found Sep 08 08:48:27.333759 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:27.333717 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-xnd5g" event={"ID":"df2da370-7d8d-414d-b82f-3d857fadd070","Type":"ContainerStarted","Data":"46acaaf7674f5f188c31b22e4445eba3454c4b745e911f90c3c9078ef361b1b7"} Sep 08 08:48:27.576899 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:27.576860 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/242ea579-b683-4fe7-a860-89724bce185b-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-kcwgm\" (UID: \"242ea579-b683-4fe7-a860-89724bce185b\") " pod="openshift-network-console/networking-console-plugin-79977f9844-kcwgm" Sep 08 08:48:27.577192 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:27.577049 2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 08 08:48:27.577192 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:27.577116 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/242ea579-b683-4fe7-a860-89724bce185b-networking-console-plugin-cert podName:242ea579-b683-4fe7-a860-89724bce185b nodeName:}" failed. No retries permitted until 2026-09-08 08:48:29.577099008 +0000 UTC m=+56.085205754 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/242ea579-b683-4fe7-a860-89724bce185b-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-kcwgm" (UID: "242ea579-b683-4fe7-a860-89724bce185b") : secret "networking-console-plugin-cert" not found Sep 08 08:48:29.595638 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:29.595594 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/242ea579-b683-4fe7-a860-89724bce185b-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-kcwgm\" (UID: \"242ea579-b683-4fe7-a860-89724bce185b\") " pod="openshift-network-console/networking-console-plugin-79977f9844-kcwgm" Sep 08 08:48:29.596085 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:29.595780 2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 08 08:48:29.596085 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:29.595860 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/242ea579-b683-4fe7-a860-89724bce185b-networking-console-plugin-cert podName:242ea579-b683-4fe7-a860-89724bce185b nodeName:}" failed. No retries permitted until 2026-09-08 08:48:33.595840013 +0000 UTC m=+60.103946766 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/242ea579-b683-4fe7-a860-89724bce185b-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-kcwgm" (UID: "242ea579-b683-4fe7-a860-89724bce185b") : secret "networking-console-plugin-cert" not found Sep 08 08:48:29.926907 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:29.926822 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-blnpx"] Sep 08 08:48:29.930177 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:29.930149 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-blnpx" Sep 08 08:48:29.932458 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:29.932435 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 08 08:48:29.932606 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:29.932581 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 08 08:48:29.932688 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:29.932623 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pjzgc\"" Sep 08 08:48:29.933162 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:29.933145 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 08 08:48:29.933242 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:29.933163 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 08 08:48:29.942215 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:29.942152 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-blnpx"] Sep 08 08:48:29.999120 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:29.999094 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-crio-socket\") pod \"insights-runtime-extractor-blnpx\" (UID: \"7555e1e8-2114-4e83-aab0-9ffa8e0a065d\") " pod="openshift-insights/insights-runtime-extractor-blnpx" Sep 08 08:48:29.999235 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:29.999133 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-blnpx\" (UID: \"7555e1e8-2114-4e83-aab0-9ffa8e0a065d\") " pod="openshift-insights/insights-runtime-extractor-blnpx" Sep 08 08:48:29.999298 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:29.999237 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-data-volume\") pod \"insights-runtime-extractor-blnpx\" (UID: \"7555e1e8-2114-4e83-aab0-9ffa8e0a065d\") " pod="openshift-insights/insights-runtime-extractor-blnpx" Sep 08 08:48:29.999298 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:29.999274 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-blnpx\" (UID: \"7555e1e8-2114-4e83-aab0-9ffa8e0a065d\") " pod="openshift-insights/insights-runtime-extractor-blnpx" Sep 08 08:48:29.999406 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:29.999301 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxpfx\" (UniqueName: \"kubernetes.io/projected/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-kube-api-access-pxpfx\") pod \"insights-runtime-extractor-blnpx\" (UID: \"7555e1e8-2114-4e83-aab0-9ffa8e0a065d\") " pod="openshift-insights/insights-runtime-extractor-blnpx" Sep 08 08:48:30.099739 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:30.099702 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-blnpx\" (UID: \"7555e1e8-2114-4e83-aab0-9ffa8e0a065d\") " pod="openshift-insights/insights-runtime-extractor-blnpx" Sep 08 08:48:30.099739 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:30.099745 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxpfx\" (UniqueName: \"kubernetes.io/projected/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-kube-api-access-pxpfx\") pod \"insights-runtime-extractor-blnpx\" (UID: \"7555e1e8-2114-4e83-aab0-9ffa8e0a065d\") " pod="openshift-insights/insights-runtime-extractor-blnpx" Sep 08 08:48:30.099942 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:30.099811 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-crio-socket\") pod \"insights-runtime-extractor-blnpx\" (UID: \"7555e1e8-2114-4e83-aab0-9ffa8e0a065d\") " pod="openshift-insights/insights-runtime-extractor-blnpx" Sep 08 08:48:30.099942 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:30.099841 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-blnpx\" (UID: \"7555e1e8-2114-4e83-aab0-9ffa8e0a065d\") " pod="openshift-insights/insights-runtime-extractor-blnpx" Sep 08 08:48:30.099942 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:30.099880 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 08 08:48:30.099942 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:30.099908 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-data-volume\") pod \"insights-runtime-extractor-blnpx\" (UID: \"7555e1e8-2114-4e83-aab0-9ffa8e0a065d\") " pod="openshift-insights/insights-runtime-extractor-blnpx" Sep 08 08:48:30.100195 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:30.099965 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-insights-runtime-extractor-tls podName:7555e1e8-2114-4e83-aab0-9ffa8e0a065d nodeName:}" failed. No retries permitted until 2026-09-08 08:48:30.599944662 +0000 UTC m=+57.108051412 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-blnpx" (UID: "7555e1e8-2114-4e83-aab0-9ffa8e0a065d") : secret "insights-runtime-extractor-tls" not found Sep 08 08:48:30.100195 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:30.100004 2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 08 08:48:30.100195 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:30.100061 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-kube-rbac-proxy-cm podName:7555e1e8-2114-4e83-aab0-9ffa8e0a065d nodeName:}" failed. No retries permitted until 2026-09-08 08:48:30.600044734 +0000 UTC m=+57.108151489 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-blnpx" (UID: "7555e1e8-2114-4e83-aab0-9ffa8e0a065d") : configmap "kube-rbac-proxy" not found Sep 08 08:48:30.100195 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:30.100054 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-crio-socket\") pod \"insights-runtime-extractor-blnpx\" (UID: \"7555e1e8-2114-4e83-aab0-9ffa8e0a065d\") " pod="openshift-insights/insights-runtime-extractor-blnpx" Sep 08 08:48:30.100371 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:30.100291 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-data-volume\") pod \"insights-runtime-extractor-blnpx\" (UID: \"7555e1e8-2114-4e83-aab0-9ffa8e0a065d\") " pod="openshift-insights/insights-runtime-extractor-blnpx" Sep 08 08:48:30.108695 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:30.108670 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxpfx\" (UniqueName: \"kubernetes.io/projected/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-kube-api-access-pxpfx\") pod \"insights-runtime-extractor-blnpx\" (UID: \"7555e1e8-2114-4e83-aab0-9ffa8e0a065d\") " pod="openshift-insights/insights-runtime-extractor-blnpx" Sep 08 08:48:30.604570 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:30.604508 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-blnpx\" (UID: \"7555e1e8-2114-4e83-aab0-9ffa8e0a065d\") " pod="openshift-insights/insights-runtime-extractor-blnpx" Sep 08 08:48:30.604948 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:30.604589 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-blnpx\" (UID: \"7555e1e8-2114-4e83-aab0-9ffa8e0a065d\") " pod="openshift-insights/insights-runtime-extractor-blnpx" Sep 08 08:48:30.604948 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:30.604662 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 08 08:48:30.604948 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:30.604716 2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 08 08:48:30.604948 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:30.604722 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-insights-runtime-extractor-tls podName:7555e1e8-2114-4e83-aab0-9ffa8e0a065d nodeName:}" failed. No retries permitted until 2026-09-08 08:48:31.604709284 +0000 UTC m=+58.112816034 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-blnpx" (UID: "7555e1e8-2114-4e83-aab0-9ffa8e0a065d") : secret "insights-runtime-extractor-tls" not found Sep 08 08:48:30.604948 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:30.604791 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-kube-rbac-proxy-cm podName:7555e1e8-2114-4e83-aab0-9ffa8e0a065d nodeName:}" failed. No retries permitted until 2026-09-08 08:48:31.604773949 +0000 UTC m=+58.112880699 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-blnpx" (UID: "7555e1e8-2114-4e83-aab0-9ffa8e0a065d") : configmap "kube-rbac-proxy" not found Sep 08 08:48:31.351698 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:31.351666 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-4zq87" event={"ID":"b81d4647-152e-4310-8b25-4624a76694e8","Type":"ContainerStarted","Data":"a1762cfca06cac6d10951e57cfc662cc77bd52fa10a2d18ab976dfe0737705d5"} Sep 08 08:48:31.353306 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:31.353274 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-v5vkr" event={"ID":"5ee267e6-6baf-499a-9e98-78e38f849123","Type":"ContainerStarted","Data":"63275bb9b010ab620521f8d67f705e1242312ca4e64cee5e4544b2ab9ab9eaec"} Sep 08 08:48:31.354922 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:31.354900 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xnd5g_df2da370-7d8d-414d-b82f-3d857fadd070/console-operator/0.log" Sep 08 08:48:31.355069 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:31.354940 2554 generic.go:356] "Generic (PLEG): container finished" podID="df2da370-7d8d-414d-b82f-3d857fadd070" containerID="e8183ed06c0ff52a98f2d2080dcc087fdbd0dd7bd04353a84ad54beb2db727dd" exitCode=255 Sep 08 08:48:31.355069 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:31.355014 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-xnd5g" event={"ID":"df2da370-7d8d-414d-b82f-3d857fadd070","Type":"ContainerDied","Data":"e8183ed06c0ff52a98f2d2080dcc087fdbd0dd7bd04353a84ad54beb2db727dd"} Sep 08 08:48:31.355221 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:31.355204 2554 scope.go:117] "RemoveContainer" containerID="e8183ed06c0ff52a98f2d2080dcc087fdbd0dd7bd04353a84ad54beb2db727dd" Sep 08 08:48:31.357040 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:31.357015 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hs6rq" event={"ID":"bd212df8-59bb-46ae-8a18-3d1eb3cbf305","Type":"ContainerStarted","Data":"471c1c73f15d928e40e05f7ae2d5e6015fc7bbf96789b122e8cbe1cd5af06bfa"} Sep 08 08:48:31.371348 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:31.371305 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-4zq87" podStartSLOduration=2.3770759249999998 podStartE2EDuration="6.371288787s" totalImagesPullingTime="3.994212862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 08:48:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:48:31.37057639 +0000 UTC m=+57.878683160" watchObservedRunningTime="2026-09-08 08:48:31.371288787 +0000 UTC m=+57.879395553" Sep 08 08:48:31.382265 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:31.382243 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-sk6kk"] Sep 08 08:48:31.407652 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:31.407624 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-sk6kk"] Sep 08 08:48:31.412688 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:31.412383 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-sk6kk" Sep 08 08:48:31.419364 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:31.419340 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Sep 08 08:48:31.420082 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:31.420054 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Sep 08 08:48:31.420280 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:31.420261 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-2kr2q\"" Sep 08 08:48:31.437201 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:31.437155 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-v5vkr" podStartSLOduration=2.332103259 podStartE2EDuration="6.437136937s" totalImagesPullingTime="4.105033678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 08:48:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:48:31.436164479 +0000 UTC m=+57.944271247" watchObservedRunningTime="2026-09-08 08:48:31.437136937 +0000 UTC m=+57.945243706" Sep 08 08:48:31.437292 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:31.437256 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hs6rq" podStartSLOduration=2.435139402 podStartE2EDuration="6.437246064s" totalImagesPullingTime="4.002106662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 08:48:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:48:31.412664335 +0000 UTC m=+57.920771103" watchObservedRunningTime="2026-09-08 08:48:31.437246064 +0000 UTC m=+57.945352833" Sep 08 08:48:31.514638 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:31.514594 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4znqg\" (UniqueName: \"kubernetes.io/projected/db100362-1340-49e9-b87f-73b3d0d415e5-kube-api-access-4znqg\") pod \"migrator-fdbb8fdff-sk6kk\" (UID: \"db100362-1340-49e9-b87f-73b3d0d415e5\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-sk6kk" Sep 08 08:48:31.615503 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:31.615477 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4znqg\" (UniqueName: \"kubernetes.io/projected/db100362-1340-49e9-b87f-73b3d0d415e5-kube-api-access-4znqg\") pod \"migrator-fdbb8fdff-sk6kk\" (UID: \"db100362-1340-49e9-b87f-73b3d0d415e5\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-sk6kk" Sep 08 08:48:31.615789 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:31.615540 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-blnpx\" (UID: \"7555e1e8-2114-4e83-aab0-9ffa8e0a065d\") " pod="openshift-insights/insights-runtime-extractor-blnpx" Sep 08 08:48:31.615789 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:31.615616 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-blnpx\" (UID: \"7555e1e8-2114-4e83-aab0-9ffa8e0a065d\") " pod="openshift-insights/insights-runtime-extractor-blnpx" Sep 08 08:48:31.615789 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:31.615685 2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 08 08:48:31.615789 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:31.615715 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 08 08:48:31.615789 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:31.615741 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-kube-rbac-proxy-cm podName:7555e1e8-2114-4e83-aab0-9ffa8e0a065d nodeName:}" failed. No retries permitted until 2026-09-08 08:48:33.615722437 +0000 UTC m=+60.123829190 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-blnpx" (UID: "7555e1e8-2114-4e83-aab0-9ffa8e0a065d") : configmap "kube-rbac-proxy" not found Sep 08 08:48:31.615789 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:31.615777 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-insights-runtime-extractor-tls podName:7555e1e8-2114-4e83-aab0-9ffa8e0a065d nodeName:}" failed. No retries permitted until 2026-09-08 08:48:33.615758404 +0000 UTC m=+60.123865156 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-blnpx" (UID: "7555e1e8-2114-4e83-aab0-9ffa8e0a065d") : secret "insights-runtime-extractor-tls" not found Sep 08 08:48:31.623444 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:31.623395 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4znqg\" (UniqueName: \"kubernetes.io/projected/db100362-1340-49e9-b87f-73b3d0d415e5-kube-api-access-4znqg\") pod \"migrator-fdbb8fdff-sk6kk\" (UID: \"db100362-1340-49e9-b87f-73b3d0d415e5\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-sk6kk" Sep 08 08:48:31.707289 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:31.707263 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lbvgv_a4db69aa-e494-477b-8263-f4c7d54f0d71/dns-node-resolver/0.log" Sep 08 08:48:31.726514 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:31.726490 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-sk6kk" Sep 08 08:48:32.009480 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:32.009452 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-sk6kk"] Sep 08 08:48:32.011942 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:48:32.011917 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb100362_1340_49e9_b87f_73b3d0d415e5.slice/crio-9a5597b7c8c23607d9bed4e45fe472abdd4cc12707260bd0c587a5f11cae57d0 WatchSource:0}: Error finding container 9a5597b7c8c23607d9bed4e45fe472abdd4cc12707260bd0c587a5f11cae57d0: Status 404 returned error can't find the container with id 9a5597b7c8c23607d9bed4e45fe472abdd4cc12707260bd0c587a5f11cae57d0 Sep 08 08:48:32.305254 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:32.305187 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8bx7f_01944a8d-4aa3-4e4b-9df4-922d4b99f848/node-ca/0.log" Sep 08 08:48:32.360867 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:32.360835 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-sk6kk" event={"ID":"db100362-1340-49e9-b87f-73b3d0d415e5","Type":"ContainerStarted","Data":"9a5597b7c8c23607d9bed4e45fe472abdd4cc12707260bd0c587a5f11cae57d0"} Sep 08 08:48:32.362144 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:32.362122 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xnd5g_df2da370-7d8d-414d-b82f-3d857fadd070/console-operator/1.log" Sep 08 08:48:32.362498 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:32.362483 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xnd5g_df2da370-7d8d-414d-b82f-3d857fadd070/console-operator/0.log" Sep 08 08:48:32.362573 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:32.362518 2554 generic.go:356] "Generic (PLEG): container finished" podID="df2da370-7d8d-414d-b82f-3d857fadd070" containerID="438f939e532c9df193b6d60afa6b823cf0b4f22d7c44e938ef5bf80b1a3bdffc" exitCode=255 Sep 08 08:48:32.362624 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:32.362598 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-xnd5g" event={"ID":"df2da370-7d8d-414d-b82f-3d857fadd070","Type":"ContainerDied","Data":"438f939e532c9df193b6d60afa6b823cf0b4f22d7c44e938ef5bf80b1a3bdffc"} Sep 08 08:48:32.362677 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:32.362644 2554 scope.go:117] "RemoveContainer" containerID="e8183ed06c0ff52a98f2d2080dcc087fdbd0dd7bd04353a84ad54beb2db727dd" Sep 08 08:48:32.362821 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:32.362803 2554 scope.go:117] "RemoveContainer" containerID="438f939e532c9df193b6d60afa6b823cf0b4f22d7c44e938ef5bf80b1a3bdffc" Sep 08 08:48:32.363011 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:32.362969 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-xnd5g_openshift-console-operator(df2da370-7d8d-414d-b82f-3d857fadd070)\"" pod="openshift-console-operator/console-operator-85b454874b-xnd5g" podUID="df2da370-7d8d-414d-b82f-3d857fadd070" Sep 08 08:48:33.105459 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:33.105430 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lp6dm_923c000a-89fd-4a8f-a8ff-5aa9a32c3509/serve-healthcheck-canary/0.log" Sep 08 08:48:33.264787 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:33.264763 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xn2lh" Sep 08 08:48:33.372217 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:33.372140 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xnd5g_df2da370-7d8d-414d-b82f-3d857fadd070/console-operator/1.log" Sep 08 08:48:33.372439 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:33.372418 2554 scope.go:117] "RemoveContainer" containerID="438f939e532c9df193b6d60afa6b823cf0b4f22d7c44e938ef5bf80b1a3bdffc" Sep 08 08:48:33.372598 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:33.372575 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-xnd5g_openshift-console-operator(df2da370-7d8d-414d-b82f-3d857fadd070)\"" pod="openshift-console-operator/console-operator-85b454874b-xnd5g" podUID="df2da370-7d8d-414d-b82f-3d857fadd070" Sep 08 08:48:33.373870 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:33.373851 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-sk6kk" event={"ID":"db100362-1340-49e9-b87f-73b3d0d415e5","Type":"ContainerStarted","Data":"ca8317614069bd72a348ef67fd408fb5b85123d33a5f860d2a680ad8996e3b94"} Sep 08 08:48:33.373957 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:33.373908 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-sk6kk" event={"ID":"db100362-1340-49e9-b87f-73b3d0d415e5","Type":"ContainerStarted","Data":"a88ed2fb953252fec0129221962569bcd737cf88b5d7110ee6a716155a1563b9"} Sep 08 08:48:33.406310 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:33.406273 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-sk6kk" podStartSLOduration=1.382589731 podStartE2EDuration="2.406260678s" totalImagesPullingTime="1.023670947s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 08:48:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:48:33.405752271 +0000 UTC m=+59.913859039" watchObservedRunningTime="2026-09-08 08:48:33.406260678 +0000 UTC m=+59.914367445" Sep 08 08:48:33.633358 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:33.633295 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-blnpx\" (UID: \"7555e1e8-2114-4e83-aab0-9ffa8e0a065d\") " pod="openshift-insights/insights-runtime-extractor-blnpx" Sep 08 08:48:33.633358 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:33.633348 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/242ea579-b683-4fe7-a860-89724bce185b-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-kcwgm\" (UID: \"242ea579-b683-4fe7-a860-89724bce185b\") " pod="openshift-network-console/networking-console-plugin-79977f9844-kcwgm" Sep 08 08:48:33.633503 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:33.633381 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-blnpx\" (UID: \"7555e1e8-2114-4e83-aab0-9ffa8e0a065d\") " pod="openshift-insights/insights-runtime-extractor-blnpx" Sep 08 08:48:33.633503 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:33.633398 2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 08 08:48:33.633503 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:33.633460 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-kube-rbac-proxy-cm podName:7555e1e8-2114-4e83-aab0-9ffa8e0a065d nodeName:}" failed. No retries permitted until 2026-09-08 08:48:37.63344125 +0000 UTC m=+64.141548007 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-blnpx" (UID: "7555e1e8-2114-4e83-aab0-9ffa8e0a065d") : configmap "kube-rbac-proxy" not found Sep 08 08:48:33.633503 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:33.633469 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 08 08:48:33.633503 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:33.633493 2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 08 08:48:33.633503 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:33.633504 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-insights-runtime-extractor-tls podName:7555e1e8-2114-4e83-aab0-9ffa8e0a065d nodeName:}" failed. No retries permitted until 2026-09-08 08:48:37.633493743 +0000 UTC m=+64.141600489 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-blnpx" (UID: "7555e1e8-2114-4e83-aab0-9ffa8e0a065d") : secret "insights-runtime-extractor-tls" not found Sep 08 08:48:33.633691 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:33.633539 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/242ea579-b683-4fe7-a860-89724bce185b-networking-console-plugin-cert podName:242ea579-b683-4fe7-a860-89724bce185b nodeName:}" failed. No retries permitted until 2026-09-08 08:48:41.633525909 +0000 UTC m=+68.141632663 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/242ea579-b683-4fe7-a860-89724bce185b-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-kcwgm" (UID: "242ea579-b683-4fe7-a860-89724bce185b") : secret "networking-console-plugin-cert" not found Sep 08 08:48:36.114795 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:36.114756 2554 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-xnd5g" Sep 08 08:48:36.115271 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:36.114808 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-xnd5g" Sep 08 08:48:36.115271 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:36.115144 2554 scope.go:117] "RemoveContainer" containerID="438f939e532c9df193b6d60afa6b823cf0b4f22d7c44e938ef5bf80b1a3bdffc" Sep 08 08:48:36.115343 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:36.115302 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-xnd5g_openshift-console-operator(df2da370-7d8d-414d-b82f-3d857fadd070)\"" pod="openshift-console-operator/console-operator-85b454874b-xnd5g" podUID="df2da370-7d8d-414d-b82f-3d857fadd070" Sep 08 08:48:37.663323 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:37.663290 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-blnpx\" (UID: \"7555e1e8-2114-4e83-aab0-9ffa8e0a065d\") " pod="openshift-insights/insights-runtime-extractor-blnpx" Sep 08 08:48:37.663681 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:37.663352 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-blnpx\" (UID: \"7555e1e8-2114-4e83-aab0-9ffa8e0a065d\") " pod="openshift-insights/insights-runtime-extractor-blnpx" Sep 08 08:48:37.663681 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:37.663407 2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 08 08:48:37.663681 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:37.663432 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 08 08:48:37.663681 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:37.663454 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-kube-rbac-proxy-cm podName:7555e1e8-2114-4e83-aab0-9ffa8e0a065d nodeName:}" failed. No retries permitted until 2026-09-08 08:48:45.663440262 +0000 UTC m=+72.171547008 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-blnpx" (UID: "7555e1e8-2114-4e83-aab0-9ffa8e0a065d") : configmap "kube-rbac-proxy" not found Sep 08 08:48:37.663681 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:37.663483 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-insights-runtime-extractor-tls podName:7555e1e8-2114-4e83-aab0-9ffa8e0a065d nodeName:}" failed. No retries permitted until 2026-09-08 08:48:45.663468776 +0000 UTC m=+72.171575522 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-blnpx" (UID: "7555e1e8-2114-4e83-aab0-9ffa8e0a065d") : secret "insights-runtime-extractor-tls" not found Sep 08 08:48:39.680692 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:39.680654 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57716ea8-fa5c-47c2-a186-7a0acbf6ae38-metrics-certs\") pod \"network-metrics-daemon-9xk7m\" (UID: \"57716ea8-fa5c-47c2-a186-7a0acbf6ae38\") " pod="openshift-multus/network-metrics-daemon-9xk7m" Sep 08 08:48:39.681118 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:39.680709 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1746be3a-25ff-4dc5-b4fb-6075f46391e0-original-pull-secret\") pod \"global-pull-secret-syncer-b6s4m\" (UID: \"1746be3a-25ff-4dc5-b4fb-6075f46391e0\") " pod="kube-system/global-pull-secret-syncer-b6s4m" Sep 08 08:48:39.681118 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:39.680852 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dq6cj\" (UniqueName: \"kubernetes.io/projected/5e400e6f-9bee-43f4-b6b1-f75ba65c39fc-kube-api-access-dq6cj\") pod \"network-check-target-z4smd\" (UID: \"5e400e6f-9bee-43f4-b6b1-f75ba65c39fc\") " pod="openshift-network-diagnostics/network-check-target-z4smd" Sep 08 08:48:39.683420 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:39.683399 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 08 08:48:39.683495 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:39.683422 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 08 08:48:39.683612 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:39.683591 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dq6cj\" (UniqueName: \"kubernetes.io/projected/5e400e6f-9bee-43f4-b6b1-f75ba65c39fc-kube-api-access-dq6cj\") pod \"network-check-target-z4smd\" (UID: \"5e400e6f-9bee-43f4-b6b1-f75ba65c39fc\") " pod="openshift-network-diagnostics/network-check-target-z4smd" Sep 08 08:48:39.691405 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:39.691386 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 08 08:48:39.691491 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:39.691443 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57716ea8-fa5c-47c2-a186-7a0acbf6ae38-metrics-certs podName:57716ea8-fa5c-47c2-a186-7a0acbf6ae38 nodeName:}" failed. No retries permitted until 2026-09-08 08:49:43.691428181 +0000 UTC m=+130.199534927 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57716ea8-fa5c-47c2-a186-7a0acbf6ae38-metrics-certs") pod "network-metrics-daemon-9xk7m" (UID: "57716ea8-fa5c-47c2-a186-7a0acbf6ae38") : secret "metrics-daemon-secret" not found Sep 08 08:48:39.693416 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:39.693393 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1746be3a-25ff-4dc5-b4fb-6075f46391e0-original-pull-secret\") pod \"global-pull-secret-syncer-b6s4m\" (UID: \"1746be3a-25ff-4dc5-b4fb-6075f46391e0\") " pod="kube-system/global-pull-secret-syncer-b6s4m" Sep 08 08:48:39.742551 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:39.742531 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b6s4m" Sep 08 08:48:39.752554 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:39.752529 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-d8k6z\"" Sep 08 08:48:39.761027 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:39.760964 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4smd" Sep 08 08:48:40.011859 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:40.011832 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-b6s4m"] Sep 08 08:48:40.014748 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:48:40.014726 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1746be3a_25ff_4dc5_b4fb_6075f46391e0.slice/crio-02f12fe32eddbfb4db0c4b6391eacdf7253d0b456d9d465415dc98ec6db176df WatchSource:0}: Error finding container 02f12fe32eddbfb4db0c4b6391eacdf7253d0b456d9d465415dc98ec6db176df: Status 404 returned error can't find the container with id 02f12fe32eddbfb4db0c4b6391eacdf7253d0b456d9d465415dc98ec6db176df Sep 08 08:48:40.037089 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:40.037064 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z4smd"] Sep 08 08:48:40.039451 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:48:40.039422 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e400e6f_9bee_43f4_b6b1_f75ba65c39fc.slice/crio-e437e57d2a275f2ead7abdef6ba3a7fa7eae30ec0213ef38a5ba549b6b6ae5b7 WatchSource:0}: Error finding container e437e57d2a275f2ead7abdef6ba3a7fa7eae30ec0213ef38a5ba549b6b6ae5b7: Status 404 returned error can't find the container with id e437e57d2a275f2ead7abdef6ba3a7fa7eae30ec0213ef38a5ba549b6b6ae5b7 Sep 08 08:48:40.396515 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:40.396432 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-b6s4m" event={"ID":"1746be3a-25ff-4dc5-b4fb-6075f46391e0","Type":"ContainerStarted","Data":"02f12fe32eddbfb4db0c4b6391eacdf7253d0b456d9d465415dc98ec6db176df"} Sep 08 08:48:40.397774 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:40.397747 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z4smd" event={"ID":"5e400e6f-9bee-43f4-b6b1-f75ba65c39fc","Type":"ContainerStarted","Data":"526bd502579a960d81ad82eb3ae0619a9c484d258151e364f93dbad19fea4437"} Sep 08 08:48:40.397886 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:40.397793 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z4smd" event={"ID":"5e400e6f-9bee-43f4-b6b1-f75ba65c39fc","Type":"ContainerStarted","Data":"e437e57d2a275f2ead7abdef6ba3a7fa7eae30ec0213ef38a5ba549b6b6ae5b7"} Sep 08 08:48:40.397934 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:40.397880 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-z4smd" Sep 08 08:48:40.415428 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:40.415388 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-z4smd" podStartSLOduration=66.41537658 podStartE2EDuration="1m6.41537658s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 08:47:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:48:40.415028205 +0000 UTC m=+66.923134970" watchObservedRunningTime="2026-09-08 08:48:40.41537658 +0000 UTC m=+66.923483347" Sep 08 08:48:40.687999 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:40.687908 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-registry-tls\") pod \"image-registry-77696969ff-xzvfk\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " pod="openshift-image-registry/image-registry-77696969ff-xzvfk" Sep 08 08:48:40.688341 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:40.688047 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/47fa42d5-9f58-489c-8031-4a366e32dfc1-metrics-tls\") pod \"dns-default-wcgdd\" (UID: \"47fa42d5-9f58-489c-8031-4a366e32dfc1\") " pod="openshift-dns/dns-default-wcgdd" Sep 08 08:48:40.688341 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:40.688061 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 08 08:48:40.688341 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:40.688078 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77696969ff-xzvfk: secret "image-registry-tls" not found Sep 08 08:48:40.688341 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:40.688129 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-registry-tls podName:c7af2e1e-49c0-4489-9f5b-32bb9d086ccd nodeName:}" failed. No retries permitted until 2026-09-08 08:49:12.688114089 +0000 UTC m=+99.196220834 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-registry-tls") pod "image-registry-77696969ff-xzvfk" (UID: "c7af2e1e-49c0-4489-9f5b-32bb9d086ccd") : secret "image-registry-tls" not found Sep 08 08:48:40.688341 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:40.688163 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 08:48:40.688341 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:40.688229 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47fa42d5-9f58-489c-8031-4a366e32dfc1-metrics-tls podName:47fa42d5-9f58-489c-8031-4a366e32dfc1 nodeName:}" failed. No retries permitted until 2026-09-08 08:49:12.68821379 +0000 UTC m=+99.196320550 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/47fa42d5-9f58-489c-8031-4a366e32dfc1-metrics-tls") pod "dns-default-wcgdd" (UID: "47fa42d5-9f58-489c-8031-4a366e32dfc1") : secret "dns-default-metrics-tls" not found Sep 08 08:48:41.696241 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:41.696205 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/242ea579-b683-4fe7-a860-89724bce185b-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-kcwgm\" (UID: \"242ea579-b683-4fe7-a860-89724bce185b\") " pod="openshift-network-console/networking-console-plugin-79977f9844-kcwgm" Sep 08 08:48:41.696679 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:41.696361 2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 08 08:48:41.696679 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:41.696423 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/242ea579-b683-4fe7-a860-89724bce185b-networking-console-plugin-cert podName:242ea579-b683-4fe7-a860-89724bce185b nodeName:}" failed. No retries permitted until 2026-09-08 08:48:57.696408349 +0000 UTC m=+84.204515094 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/242ea579-b683-4fe7-a860-89724bce185b-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-kcwgm" (UID: "242ea579-b683-4fe7-a860-89724bce185b") : secret "networking-console-plugin-cert" not found Sep 08 08:48:45.414687 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:45.414651 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-b6s4m" event={"ID":"1746be3a-25ff-4dc5-b4fb-6075f46391e0","Type":"ContainerStarted","Data":"ef76a3d2ed91e88d6e322b1b7ef0e8344324aced6e9f30e3117f0c3b3ac92309"} Sep 08 08:48:45.429226 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:45.429181 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-b6s4m" podStartSLOduration=66.766341635 podStartE2EDuration="1m11.429168397s" totalImagesPullingTime="4.662826762s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 08:47:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:48:45.428570361 +0000 UTC m=+71.936677130" watchObservedRunningTime="2026-09-08 08:48:45.429168397 +0000 UTC m=+71.937275165" Sep 08 08:48:45.729345 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:45.729321 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-blnpx\" (UID: \"7555e1e8-2114-4e83-aab0-9ffa8e0a065d\") " pod="openshift-insights/insights-runtime-extractor-blnpx" Sep 08 08:48:45.729485 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:45.729374 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-blnpx\" (UID: \"7555e1e8-2114-4e83-aab0-9ffa8e0a065d\") " pod="openshift-insights/insights-runtime-extractor-blnpx" Sep 08 08:48:45.729485 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:45.729467 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 08 08:48:45.729568 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:45.729542 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-insights-runtime-extractor-tls podName:7555e1e8-2114-4e83-aab0-9ffa8e0a065d nodeName:}" failed. No retries permitted until 2026-09-08 08:49:01.729525477 +0000 UTC m=+88.237632228 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-blnpx" (UID: "7555e1e8-2114-4e83-aab0-9ffa8e0a065d") : secret "insights-runtime-extractor-tls" not found Sep 08 08:48:45.729907 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:45.729888 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-blnpx\" (UID: \"7555e1e8-2114-4e83-aab0-9ffa8e0a065d\") " pod="openshift-insights/insights-runtime-extractor-blnpx" Sep 08 08:48:47.027225 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:47.027195 2554 scope.go:117] "RemoveContainer" containerID="438f939e532c9df193b6d60afa6b823cf0b4f22d7c44e938ef5bf80b1a3bdffc" Sep 08 08:48:47.422103 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:47.422029 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xnd5g_df2da370-7d8d-414d-b82f-3d857fadd070/console-operator/2.log" Sep 08 08:48:47.422434 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:47.422414 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xnd5g_df2da370-7d8d-414d-b82f-3d857fadd070/console-operator/1.log" Sep 08 08:48:47.422541 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:47.422452 2554 generic.go:356] "Generic (PLEG): container finished" podID="df2da370-7d8d-414d-b82f-3d857fadd070" containerID="09a649711655a6771a6220122114685eaf09edd0abcdff0c196816538f2746cf" exitCode=255 Sep 08 08:48:47.422541 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:47.422499 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-xnd5g" event={"ID":"df2da370-7d8d-414d-b82f-3d857fadd070","Type":"ContainerDied","Data":"09a649711655a6771a6220122114685eaf09edd0abcdff0c196816538f2746cf"} Sep 08 08:48:47.422654 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:47.422557 2554 scope.go:117] "RemoveContainer" containerID="438f939e532c9df193b6d60afa6b823cf0b4f22d7c44e938ef5bf80b1a3bdffc" Sep 08 08:48:47.422845 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:47.422828 2554 scope.go:117] "RemoveContainer" containerID="09a649711655a6771a6220122114685eaf09edd0abcdff0c196816538f2746cf" Sep 08 08:48:47.423044 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:47.423024 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-xnd5g_openshift-console-operator(df2da370-7d8d-414d-b82f-3d857fadd070)\"" pod="openshift-console-operator/console-operator-85b454874b-xnd5g" podUID="df2da370-7d8d-414d-b82f-3d857fadd070" Sep 08 08:48:48.427816 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:48.427789 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xnd5g_df2da370-7d8d-414d-b82f-3d857fadd070/console-operator/2.log" Sep 08 08:48:56.114519 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:56.114481 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-xnd5g" Sep 08 08:48:56.115018 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:56.114541 2554 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-xnd5g" Sep 08 08:48:56.115018 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:56.114880 2554 scope.go:117] "RemoveContainer" containerID="09a649711655a6771a6220122114685eaf09edd0abcdff0c196816538f2746cf" Sep 08 08:48:56.115139 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:48:56.115083 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-xnd5g_openshift-console-operator(df2da370-7d8d-414d-b82f-3d857fadd070)\"" pod="openshift-console-operator/console-operator-85b454874b-xnd5g" podUID="df2da370-7d8d-414d-b82f-3d857fadd070" Sep 08 08:48:57.714402 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:57.714365 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/242ea579-b683-4fe7-a860-89724bce185b-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-kcwgm\" (UID: \"242ea579-b683-4fe7-a860-89724bce185b\") " pod="openshift-network-console/networking-console-plugin-79977f9844-kcwgm" Sep 08 08:48:57.716731 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:57.716703 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/242ea579-b683-4fe7-a860-89724bce185b-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-kcwgm\" (UID: \"242ea579-b683-4fe7-a860-89724bce185b\") " pod="openshift-network-console/networking-console-plugin-79977f9844-kcwgm" Sep 08 08:48:58.004785 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:58.004709 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-7775r\"" Sep 08 08:48:58.012640 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:58.012617 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-kcwgm" Sep 08 08:48:58.281286 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:58.281200 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-kcwgm"] Sep 08 08:48:58.286150 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:48:58.286122 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod242ea579_b683_4fe7_a860_89724bce185b.slice/crio-58411142c692d1226b496eaefab088a6d16f6048611547ebb4aeb98f4d0607f9 WatchSource:0}: Error finding container 58411142c692d1226b496eaefab088a6d16f6048611547ebb4aeb98f4d0607f9: Status 404 returned error can't find the container with id 58411142c692d1226b496eaefab088a6d16f6048611547ebb4aeb98f4d0607f9 Sep 08 08:48:58.459144 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:48:58.459106 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-kcwgm" event={"ID":"242ea579-b683-4fe7-a860-89724bce185b","Type":"ContainerStarted","Data":"58411142c692d1226b496eaefab088a6d16f6048611547ebb4aeb98f4d0607f9"} Sep 08 08:49:00.116079 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:00.116049 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gpzzn"] Sep 08 08:49:00.118081 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:00.118065 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gpzzn" Sep 08 08:49:00.120025 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:00.119996 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Sep 08 08:49:00.120240 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:00.120218 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-vbxf2\"" Sep 08 08:49:00.126917 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:00.126896 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gpzzn"] Sep 08 08:49:00.233829 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:00.233805 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/caf19724-9691-48b9-8c53-c5c908cd8bdc-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-gpzzn\" (UID: \"caf19724-9691-48b9-8c53-c5c908cd8bdc\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gpzzn" Sep 08 08:49:00.334238 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:00.334211 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/caf19724-9691-48b9-8c53-c5c908cd8bdc-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-gpzzn\" (UID: \"caf19724-9691-48b9-8c53-c5c908cd8bdc\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gpzzn" Sep 08 08:49:00.334376 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:49:00.334357 2554 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Sep 08 08:49:00.334434 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:49:00.334424 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/caf19724-9691-48b9-8c53-c5c908cd8bdc-tls-certificates podName:caf19724-9691-48b9-8c53-c5c908cd8bdc nodeName:}" failed. No retries permitted until 2026-09-08 08:49:00.834408576 +0000 UTC m=+87.342515322 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/caf19724-9691-48b9-8c53-c5c908cd8bdc-tls-certificates") pod "prometheus-operator-admission-webhook-78f599db79-gpzzn" (UID: "caf19724-9691-48b9-8c53-c5c908cd8bdc") : secret "prometheus-operator-admission-webhook-tls" not found Sep 08 08:49:00.466964 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:00.466937 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-kcwgm" event={"ID":"242ea579-b683-4fe7-a860-89724bce185b","Type":"ContainerStarted","Data":"cbe11dc19cc54be73288b2268c5f4ad241b8c3af1d8bc63ca3d4b60fe8e30016"} Sep 08 08:49:00.483252 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:00.483210 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-kcwgm" podStartSLOduration=34.330095493 podStartE2EDuration="35.483192604s" totalImagesPullingTime="1.153097111s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 08:48:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:49:00.483040744 +0000 UTC m=+86.991147515" watchObservedRunningTime="2026-09-08 08:49:00.483192604 +0000 UTC m=+86.991299374" Sep 08 08:49:00.838208 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:00.838132 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/caf19724-9691-48b9-8c53-c5c908cd8bdc-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-gpzzn\" (UID: \"caf19724-9691-48b9-8c53-c5c908cd8bdc\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gpzzn" Sep 08 08:49:00.840504 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:00.840481 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/caf19724-9691-48b9-8c53-c5c908cd8bdc-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-gpzzn\" (UID: \"caf19724-9691-48b9-8c53-c5c908cd8bdc\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gpzzn" Sep 08 08:49:01.026863 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:01.026839 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gpzzn" Sep 08 08:49:01.290143 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:01.290109 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gpzzn"] Sep 08 08:49:01.294910 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:49:01.294864 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaf19724_9691_48b9_8c53_c5c908cd8bdc.slice/crio-b44ecc6df048593b5b6afe6e8013ca8496beec6d8b42b796c788c1e4d9aaf731 WatchSource:0}: Error finding container b44ecc6df048593b5b6afe6e8013ca8496beec6d8b42b796c788c1e4d9aaf731: Status 404 returned error can't find the container with id b44ecc6df048593b5b6afe6e8013ca8496beec6d8b42b796c788c1e4d9aaf731 Sep 08 08:49:01.471882 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:01.471846 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gpzzn" event={"ID":"caf19724-9691-48b9-8c53-c5c908cd8bdc","Type":"ContainerStarted","Data":"b44ecc6df048593b5b6afe6e8013ca8496beec6d8b42b796c788c1e4d9aaf731"} Sep 08 08:49:01.744161 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:01.744128 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-blnpx\" (UID: \"7555e1e8-2114-4e83-aab0-9ffa8e0a065d\") " pod="openshift-insights/insights-runtime-extractor-blnpx" Sep 08 08:49:01.746490 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:01.746465 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7555e1e8-2114-4e83-aab0-9ffa8e0a065d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-blnpx\" (UID: \"7555e1e8-2114-4e83-aab0-9ffa8e0a065d\") " pod="openshift-insights/insights-runtime-extractor-blnpx" Sep 08 08:49:02.043207 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:02.043131 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pjzgc\"" Sep 08 08:49:02.051060 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:02.051037 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-blnpx" Sep 08 08:49:02.336688 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:02.336607 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-blnpx"] Sep 08 08:49:02.411942 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:49:02.411907 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7555e1e8_2114_4e83_aab0_9ffa8e0a065d.slice/crio-da4ffac94a1a4c19b89e3d8ed0b2cae3f1027805edf13a1c0b3bdff2682efa2d WatchSource:0}: Error finding container da4ffac94a1a4c19b89e3d8ed0b2cae3f1027805edf13a1c0b3bdff2682efa2d: Status 404 returned error can't find the container with id da4ffac94a1a4c19b89e3d8ed0b2cae3f1027805edf13a1c0b3bdff2682efa2d Sep 08 08:49:02.476238 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:02.476206 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-blnpx" event={"ID":"7555e1e8-2114-4e83-aab0-9ffa8e0a065d","Type":"ContainerStarted","Data":"da4ffac94a1a4c19b89e3d8ed0b2cae3f1027805edf13a1c0b3bdff2682efa2d"} Sep 08 08:49:03.480648 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:03.480604 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-blnpx" event={"ID":"7555e1e8-2114-4e83-aab0-9ffa8e0a065d","Type":"ContainerStarted","Data":"2128e5fb6703aebaeedaddc5ec5bf7fe2f8c5f9385fa0f92c25a0a2a8ac724bd"} Sep 08 08:49:03.482027 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:03.481959 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gpzzn" event={"ID":"caf19724-9691-48b9-8c53-c5c908cd8bdc","Type":"ContainerStarted","Data":"9817510814df0e1fc9e5925ddf275eb05c42b7cb96c6353b6be6a41fde9a22a7"} Sep 08 08:49:03.482404 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:03.482370 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gpzzn" Sep 08 08:49:03.486817 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:03.486784 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gpzzn" Sep 08 08:49:03.514060 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:03.514018 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gpzzn" podStartSLOduration=2.062699643 podStartE2EDuration="3.51400612s" totalImagesPullingTime="1.451306477s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 08:49:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:49:03.513317208 +0000 UTC m=+90.021423975" watchObservedRunningTime="2026-09-08 08:49:03.51400612 +0000 UTC m=+90.022112885" Sep 08 08:49:04.188664 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:04.188592 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-slgg5"] Sep 08 08:49:04.209506 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:04.209482 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-slgg5"] Sep 08 08:49:04.209616 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:04.209571 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-slgg5" Sep 08 08:49:04.212799 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:04.212778 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 08 08:49:04.212920 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:04.212839 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 08 08:49:04.213041 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:04.213029 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 08 08:49:04.213604 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:04.213587 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 08 08:49:04.213698 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:04.213614 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 08 08:49:04.213865 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:04.213849 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-j4v7p\"" Sep 08 08:49:04.365940 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:04.365906 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1aa1093f-f3ec-4854-81f6-a23ce51ff061-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-slgg5\" (UID: \"1aa1093f-f3ec-4854-81f6-a23ce51ff061\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-slgg5" Sep 08 08:49:04.366087 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:04.366038 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1aa1093f-f3ec-4854-81f6-a23ce51ff061-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-slgg5\" (UID: \"1aa1093f-f3ec-4854-81f6-a23ce51ff061\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-slgg5" Sep 08 08:49:04.366087 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:04.366059 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chfx4\" (UniqueName: \"kubernetes.io/projected/1aa1093f-f3ec-4854-81f6-a23ce51ff061-kube-api-access-chfx4\") pod \"prometheus-operator-6f4946db56-slgg5\" (UID: \"1aa1093f-f3ec-4854-81f6-a23ce51ff061\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-slgg5" Sep 08 08:49:04.366167 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:04.366091 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1aa1093f-f3ec-4854-81f6-a23ce51ff061-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-slgg5\" (UID: \"1aa1093f-f3ec-4854-81f6-a23ce51ff061\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-slgg5" Sep 08 08:49:04.466921 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:04.466894 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1aa1093f-f3ec-4854-81f6-a23ce51ff061-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-slgg5\" (UID: \"1aa1093f-f3ec-4854-81f6-a23ce51ff061\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-slgg5" Sep 08 08:49:04.466921 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:04.466923 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chfx4\" (UniqueName: \"kubernetes.io/projected/1aa1093f-f3ec-4854-81f6-a23ce51ff061-kube-api-access-chfx4\") pod \"prometheus-operator-6f4946db56-slgg5\" (UID: \"1aa1093f-f3ec-4854-81f6-a23ce51ff061\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-slgg5" Sep 08 08:49:04.467147 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:04.466953 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1aa1093f-f3ec-4854-81f6-a23ce51ff061-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-slgg5\" (UID: \"1aa1093f-f3ec-4854-81f6-a23ce51ff061\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-slgg5" Sep 08 08:49:04.467147 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:04.467012 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1aa1093f-f3ec-4854-81f6-a23ce51ff061-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-slgg5\" (UID: \"1aa1093f-f3ec-4854-81f6-a23ce51ff061\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-slgg5" Sep 08 08:49:04.467729 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:04.467702 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1aa1093f-f3ec-4854-81f6-a23ce51ff061-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-slgg5\" (UID: \"1aa1093f-f3ec-4854-81f6-a23ce51ff061\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-slgg5" Sep 08 08:49:04.469717 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:04.469692 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1aa1093f-f3ec-4854-81f6-a23ce51ff061-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-slgg5\" (UID: \"1aa1093f-f3ec-4854-81f6-a23ce51ff061\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-slgg5" Sep 08 08:49:04.469818 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:04.469798 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1aa1093f-f3ec-4854-81f6-a23ce51ff061-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-slgg5\" (UID: \"1aa1093f-f3ec-4854-81f6-a23ce51ff061\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-slgg5" Sep 08 08:49:04.476916 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:04.476891 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chfx4\" (UniqueName: \"kubernetes.io/projected/1aa1093f-f3ec-4854-81f6-a23ce51ff061-kube-api-access-chfx4\") pod \"prometheus-operator-6f4946db56-slgg5\" (UID: \"1aa1093f-f3ec-4854-81f6-a23ce51ff061\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-slgg5" Sep 08 08:49:04.488477 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:04.488448 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-blnpx" event={"ID":"7555e1e8-2114-4e83-aab0-9ffa8e0a065d","Type":"ContainerStarted","Data":"766f84cb09dbd59015b834666363c214fbb49378bd9b463c5c43c64744ba1e49"} Sep 08 08:49:04.518480 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:04.518461 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-slgg5" Sep 08 08:49:04.817354 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:04.817327 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-slgg5"] Sep 08 08:49:04.818693 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:49:04.818665 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aa1093f_f3ec_4854_81f6_a23ce51ff061.slice/crio-8a72b1a06b3fe456b54e1e902aaa0f065e1abe3e75d030ae4336ef63ffe896e8 WatchSource:0}: Error finding container 8a72b1a06b3fe456b54e1e902aaa0f065e1abe3e75d030ae4336ef63ffe896e8: Status 404 returned error can't find the container with id 8a72b1a06b3fe456b54e1e902aaa0f065e1abe3e75d030ae4336ef63ffe896e8 Sep 08 08:49:05.493218 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:05.493181 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-slgg5" event={"ID":"1aa1093f-f3ec-4854-81f6-a23ce51ff061","Type":"ContainerStarted","Data":"8a72b1a06b3fe456b54e1e902aaa0f065e1abe3e75d030ae4336ef63ffe896e8"} Sep 08 08:49:06.498869 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:06.498834 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-blnpx" event={"ID":"7555e1e8-2114-4e83-aab0-9ffa8e0a065d","Type":"ContainerStarted","Data":"ef10e7ce1865fe27b545c80efbed5437c06c85aec2683fd2545bb59f7a454f74"} Sep 08 08:49:06.527325 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:06.527277 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-blnpx" podStartSLOduration=34.514536251 podStartE2EDuration="37.52726168s" totalImagesPullingTime="3.012725429s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 08:48:29 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:49:06.525425031 +0000 UTC m=+93.033531799" watchObservedRunningTime="2026-09-08 08:49:06.52726168 +0000 UTC m=+93.035368448" Sep 08 08:49:07.027412 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:07.027374 2554 scope.go:117] "RemoveContainer" containerID="09a649711655a6771a6220122114685eaf09edd0abcdff0c196816538f2746cf" Sep 08 08:49:07.027585 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:49:07.027565 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-xnd5g_openshift-console-operator(df2da370-7d8d-414d-b82f-3d857fadd070)\"" pod="openshift-console-operator/console-operator-85b454874b-xnd5g" podUID="df2da370-7d8d-414d-b82f-3d857fadd070" Sep 08 08:49:07.504324 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:07.504297 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-slgg5" event={"ID":"1aa1093f-f3ec-4854-81f6-a23ce51ff061","Type":"ContainerStarted","Data":"67bd2b2ef714b018762b3f2c36c2b1b5928b90e17c54b8953c77cfc5a4ae7256"} Sep 08 08:49:08.509104 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:08.509073 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-slgg5" event={"ID":"1aa1093f-f3ec-4854-81f6-a23ce51ff061","Type":"ContainerStarted","Data":"1b45879f32e78f8a25bf43d7df942fd6fc41515afc92d759e197b8cf2b305516"} Sep 08 08:49:08.529092 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:08.529048 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-slgg5" podStartSLOduration=1.935654137 podStartE2EDuration="4.529033892s" totalImagesPullingTime="2.593379755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 08:49:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:49:08.52710597 +0000 UTC m=+95.035212739" watchObservedRunningTime="2026-09-08 08:49:08.529033892 +0000 UTC m=+95.037140657" Sep 08 08:49:10.603546 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.603514 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-zb8rb"] Sep 08 08:49:10.605864 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.605848 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zb8rb" Sep 08 08:49:10.611094 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.611070 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 08 08:49:10.611224 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.611095 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 08 08:49:10.619420 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.619401 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-78c9t\"" Sep 08 08:49:10.626011 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.625988 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-zb8rb"] Sep 08 08:49:10.659995 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.659965 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-hqrm8"] Sep 08 08:49:10.662222 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.662204 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqrm8" Sep 08 08:49:10.666240 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.666219 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Sep 08 08:49:10.666502 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.666489 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Sep 08 08:49:10.666618 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.666604 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-xzq6w\"" Sep 08 08:49:10.666800 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.666782 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Sep 08 08:49:10.685940 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.685922 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-hqrm8"] Sep 08 08:49:10.686987 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.686959 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-nlk9v"] Sep 08 08:49:10.689303 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.689288 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nlk9v" Sep 08 08:49:10.692256 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.692232 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-ct9mh\"" Sep 08 08:49:10.694594 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.694576 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 08 08:49:10.694793 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.694774 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 08 08:49:10.695071 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.695053 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 08 08:49:10.714003 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.713968 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m98gn\" (UniqueName: \"kubernetes.io/projected/cb167784-18d1-42da-9020-85970a1e6fdd-kube-api-access-m98gn\") pod \"openshift-state-metrics-657fd6c54-zb8rb\" (UID: \"cb167784-18d1-42da-9020-85970a1e6fdd\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zb8rb" Sep 08 08:49:10.714073 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.714007 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cb167784-18d1-42da-9020-85970a1e6fdd-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-zb8rb\" (UID: \"cb167784-18d1-42da-9020-85970a1e6fdd\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zb8rb" Sep 08 08:49:10.714073 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.714027 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb167784-18d1-42da-9020-85970a1e6fdd-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-zb8rb\" (UID: \"cb167784-18d1-42da-9020-85970a1e6fdd\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zb8rb" Sep 08 08:49:10.714148 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.714085 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cb167784-18d1-42da-9020-85970a1e6fdd-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-zb8rb\" (UID: \"cb167784-18d1-42da-9020-85970a1e6fdd\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zb8rb" Sep 08 08:49:10.814651 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.814622 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a2fe8108-3de7-48bb-b768-117699ca6414-root\") pod \"node-exporter-nlk9v\" (UID: \"a2fe8108-3de7-48bb-b768-117699ca6414\") " pod="openshift-monitoring/node-exporter-nlk9v" Sep 08 08:49:10.814772 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.814657 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb167784-18d1-42da-9020-85970a1e6fdd-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-zb8rb\" (UID: \"cb167784-18d1-42da-9020-85970a1e6fdd\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zb8rb" Sep 08 08:49:10.814772 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.814684 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/925bdd29-aa93-45e4-a301-e74835b7f114-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-hqrm8\" (UID: \"925bdd29-aa93-45e4-a301-e74835b7f114\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqrm8" Sep 08 08:49:10.814772 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.814715 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmsrk\" (UniqueName: \"kubernetes.io/projected/925bdd29-aa93-45e4-a301-e74835b7f114-kube-api-access-cmsrk\") pod \"kube-state-metrics-74cb57df7-hqrm8\" (UID: \"925bdd29-aa93-45e4-a301-e74835b7f114\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqrm8" Sep 08 08:49:10.814880 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.814828 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a2fe8108-3de7-48bb-b768-117699ca6414-node-exporter-wtmp\") pod \"node-exporter-nlk9v\" (UID: \"a2fe8108-3de7-48bb-b768-117699ca6414\") " pod="openshift-monitoring/node-exporter-nlk9v" Sep 08 08:49:10.814880 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.814854 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2fe8108-3de7-48bb-b768-117699ca6414-metrics-client-ca\") pod \"node-exporter-nlk9v\" (UID: \"a2fe8108-3de7-48bb-b768-117699ca6414\") " pod="openshift-monitoring/node-exporter-nlk9v" Sep 08 08:49:10.814963 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.814890 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/925bdd29-aa93-45e4-a301-e74835b7f114-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-hqrm8\" (UID: \"925bdd29-aa93-45e4-a301-e74835b7f114\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqrm8" Sep 08 08:49:10.814963 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.814940 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cb167784-18d1-42da-9020-85970a1e6fdd-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-zb8rb\" (UID: \"cb167784-18d1-42da-9020-85970a1e6fdd\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zb8rb" Sep 08 08:49:10.815059 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.814996 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2fe8108-3de7-48bb-b768-117699ca6414-sys\") pod \"node-exporter-nlk9v\" (UID: \"a2fe8108-3de7-48bb-b768-117699ca6414\") " pod="openshift-monitoring/node-exporter-nlk9v" Sep 08 08:49:10.815103 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.815083 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a2fe8108-3de7-48bb-b768-117699ca6414-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nlk9v\" (UID: \"a2fe8108-3de7-48bb-b768-117699ca6414\") " pod="openshift-monitoring/node-exporter-nlk9v" Sep 08 08:49:10.815165 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.815109 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a2fe8108-3de7-48bb-b768-117699ca6414-node-exporter-accelerators-collector-config\") pod \"node-exporter-nlk9v\" (UID: \"a2fe8108-3de7-48bb-b768-117699ca6414\") " pod="openshift-monitoring/node-exporter-nlk9v" Sep 08 08:49:10.815165 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.815134 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/925bdd29-aa93-45e4-a301-e74835b7f114-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-hqrm8\" (UID: \"925bdd29-aa93-45e4-a301-e74835b7f114\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqrm8" Sep 08 08:49:10.815165 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.815150 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/925bdd29-aa93-45e4-a301-e74835b7f114-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-hqrm8\" (UID: \"925bdd29-aa93-45e4-a301-e74835b7f114\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqrm8" Sep 08 08:49:10.815303 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.815203 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m98gn\" (UniqueName: \"kubernetes.io/projected/cb167784-18d1-42da-9020-85970a1e6fdd-kube-api-access-m98gn\") pod \"openshift-state-metrics-657fd6c54-zb8rb\" (UID: \"cb167784-18d1-42da-9020-85970a1e6fdd\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zb8rb" Sep 08 08:49:10.815303 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.815260 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a2fe8108-3de7-48bb-b768-117699ca6414-node-exporter-textfile\") pod \"node-exporter-nlk9v\" (UID: \"a2fe8108-3de7-48bb-b768-117699ca6414\") " pod="openshift-monitoring/node-exporter-nlk9v" Sep 08 08:49:10.815404 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.815301 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-schx9\" (UniqueName: \"kubernetes.io/projected/a2fe8108-3de7-48bb-b768-117699ca6414-kube-api-access-schx9\") pod \"node-exporter-nlk9v\" (UID: \"a2fe8108-3de7-48bb-b768-117699ca6414\") " pod="openshift-monitoring/node-exporter-nlk9v" Sep 08 08:49:10.815404 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.815325 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/925bdd29-aa93-45e4-a301-e74835b7f114-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-hqrm8\" (UID: \"925bdd29-aa93-45e4-a301-e74835b7f114\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqrm8" Sep 08 08:49:10.815404 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.815382 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cb167784-18d1-42da-9020-85970a1e6fdd-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-zb8rb\" (UID: \"cb167784-18d1-42da-9020-85970a1e6fdd\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zb8rb" Sep 08 08:49:10.815550 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.815420 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a2fe8108-3de7-48bb-b768-117699ca6414-node-exporter-tls\") pod \"node-exporter-nlk9v\" (UID: \"a2fe8108-3de7-48bb-b768-117699ca6414\") " pod="openshift-monitoring/node-exporter-nlk9v" Sep 08 08:49:10.815638 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.815616 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cb167784-18d1-42da-9020-85970a1e6fdd-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-zb8rb\" (UID: \"cb167784-18d1-42da-9020-85970a1e6fdd\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zb8rb" Sep 08 08:49:10.817246 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.817220 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb167784-18d1-42da-9020-85970a1e6fdd-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-zb8rb\" (UID: \"cb167784-18d1-42da-9020-85970a1e6fdd\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zb8rb" Sep 08 08:49:10.817511 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.817492 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cb167784-18d1-42da-9020-85970a1e6fdd-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-zb8rb\" (UID: \"cb167784-18d1-42da-9020-85970a1e6fdd\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zb8rb" Sep 08 08:49:10.830677 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.830656 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m98gn\" (UniqueName: \"kubernetes.io/projected/cb167784-18d1-42da-9020-85970a1e6fdd-kube-api-access-m98gn\") pod \"openshift-state-metrics-657fd6c54-zb8rb\" (UID: \"cb167784-18d1-42da-9020-85970a1e6fdd\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zb8rb" Sep 08 08:49:10.914351 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.914304 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zb8rb" Sep 08 08:49:10.916255 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.916221 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2fe8108-3de7-48bb-b768-117699ca6414-sys\") pod \"node-exporter-nlk9v\" (UID: \"a2fe8108-3de7-48bb-b768-117699ca6414\") " pod="openshift-monitoring/node-exporter-nlk9v" Sep 08 08:49:10.916349 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.916279 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a2fe8108-3de7-48bb-b768-117699ca6414-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nlk9v\" (UID: \"a2fe8108-3de7-48bb-b768-117699ca6414\") " pod="openshift-monitoring/node-exporter-nlk9v" Sep 08 08:49:10.916349 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.916298 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a2fe8108-3de7-48bb-b768-117699ca6414-node-exporter-accelerators-collector-config\") pod \"node-exporter-nlk9v\" (UID: \"a2fe8108-3de7-48bb-b768-117699ca6414\") " pod="openshift-monitoring/node-exporter-nlk9v" Sep 08 08:49:10.916349 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.916330 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/925bdd29-aa93-45e4-a301-e74835b7f114-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-hqrm8\" (UID: \"925bdd29-aa93-45e4-a301-e74835b7f114\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqrm8" Sep 08 08:49:10.916506 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.916354 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2fe8108-3de7-48bb-b768-117699ca6414-sys\") pod \"node-exporter-nlk9v\" (UID: \"a2fe8108-3de7-48bb-b768-117699ca6414\") " pod="openshift-monitoring/node-exporter-nlk9v" Sep 08 08:49:10.916506 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.916357 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/925bdd29-aa93-45e4-a301-e74835b7f114-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-hqrm8\" (UID: \"925bdd29-aa93-45e4-a301-e74835b7f114\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqrm8" Sep 08 08:49:10.916599 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.916504 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a2fe8108-3de7-48bb-b768-117699ca6414-node-exporter-textfile\") pod \"node-exporter-nlk9v\" (UID: \"a2fe8108-3de7-48bb-b768-117699ca6414\") " pod="openshift-monitoring/node-exporter-nlk9v" Sep 08 08:49:10.916599 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.916536 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-schx9\" (UniqueName: \"kubernetes.io/projected/a2fe8108-3de7-48bb-b768-117699ca6414-kube-api-access-schx9\") pod \"node-exporter-nlk9v\" (UID: \"a2fe8108-3de7-48bb-b768-117699ca6414\") " pod="openshift-monitoring/node-exporter-nlk9v" Sep 08 08:49:10.916599 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.916558 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/925bdd29-aa93-45e4-a301-e74835b7f114-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-hqrm8\" (UID: \"925bdd29-aa93-45e4-a301-e74835b7f114\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqrm8" Sep 08 08:49:10.917192 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.916647 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a2fe8108-3de7-48bb-b768-117699ca6414-node-exporter-tls\") pod \"node-exporter-nlk9v\" (UID: \"a2fe8108-3de7-48bb-b768-117699ca6414\") " pod="openshift-monitoring/node-exporter-nlk9v" Sep 08 08:49:10.917192 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.916684 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a2fe8108-3de7-48bb-b768-117699ca6414-root\") pod \"node-exporter-nlk9v\" (UID: \"a2fe8108-3de7-48bb-b768-117699ca6414\") " pod="openshift-monitoring/node-exporter-nlk9v" Sep 08 08:49:10.917192 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.916743 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/925bdd29-aa93-45e4-a301-e74835b7f114-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-hqrm8\" (UID: \"925bdd29-aa93-45e4-a301-e74835b7f114\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqrm8" Sep 08 08:49:10.917192 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.916862 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/925bdd29-aa93-45e4-a301-e74835b7f114-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-hqrm8\" (UID: \"925bdd29-aa93-45e4-a301-e74835b7f114\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqrm8" Sep 08 08:49:10.917192 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.916920 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a2fe8108-3de7-48bb-b768-117699ca6414-root\") pod \"node-exporter-nlk9v\" (UID: \"a2fe8108-3de7-48bb-b768-117699ca6414\") " pod="openshift-monitoring/node-exporter-nlk9v" Sep 08 08:49:10.917192 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.916956 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmsrk\" (UniqueName: \"kubernetes.io/projected/925bdd29-aa93-45e4-a301-e74835b7f114-kube-api-access-cmsrk\") pod \"kube-state-metrics-74cb57df7-hqrm8\" (UID: \"925bdd29-aa93-45e4-a301-e74835b7f114\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqrm8" Sep 08 08:49:10.917192 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.916960 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a2fe8108-3de7-48bb-b768-117699ca6414-node-exporter-accelerators-collector-config\") pod \"node-exporter-nlk9v\" (UID: \"a2fe8108-3de7-48bb-b768-117699ca6414\") " pod="openshift-monitoring/node-exporter-nlk9v" Sep 08 08:49:10.917192 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.917026 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a2fe8108-3de7-48bb-b768-117699ca6414-node-exporter-wtmp\") pod \"node-exporter-nlk9v\" (UID: \"a2fe8108-3de7-48bb-b768-117699ca6414\") " pod="openshift-monitoring/node-exporter-nlk9v" Sep 08 08:49:10.917192 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.917055 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2fe8108-3de7-48bb-b768-117699ca6414-metrics-client-ca\") pod \"node-exporter-nlk9v\" (UID: \"a2fe8108-3de7-48bb-b768-117699ca6414\") " pod="openshift-monitoring/node-exporter-nlk9v" Sep 08 08:49:10.917192 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.917085 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/925bdd29-aa93-45e4-a301-e74835b7f114-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-hqrm8\" (UID: \"925bdd29-aa93-45e4-a301-e74835b7f114\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqrm8" Sep 08 08:49:10.917192 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.917187 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/925bdd29-aa93-45e4-a301-e74835b7f114-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-hqrm8\" (UID: \"925bdd29-aa93-45e4-a301-e74835b7f114\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqrm8" Sep 08 08:49:10.917728 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.917208 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a2fe8108-3de7-48bb-b768-117699ca6414-node-exporter-textfile\") pod \"node-exporter-nlk9v\" (UID: \"a2fe8108-3de7-48bb-b768-117699ca6414\") " pod="openshift-monitoring/node-exporter-nlk9v" Sep 08 08:49:10.917728 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.917263 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/925bdd29-aa93-45e4-a301-e74835b7f114-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-hqrm8\" (UID: \"925bdd29-aa93-45e4-a301-e74835b7f114\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqrm8" Sep 08 08:49:10.917728 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.917345 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a2fe8108-3de7-48bb-b768-117699ca6414-node-exporter-wtmp\") pod \"node-exporter-nlk9v\" (UID: \"a2fe8108-3de7-48bb-b768-117699ca6414\") " pod="openshift-monitoring/node-exporter-nlk9v" Sep 08 08:49:10.917728 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.917650 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2fe8108-3de7-48bb-b768-117699ca6414-metrics-client-ca\") pod \"node-exporter-nlk9v\" (UID: \"a2fe8108-3de7-48bb-b768-117699ca6414\") " pod="openshift-monitoring/node-exporter-nlk9v" Sep 08 08:49:10.919295 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.919274 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a2fe8108-3de7-48bb-b768-117699ca6414-node-exporter-tls\") pod \"node-exporter-nlk9v\" (UID: \"a2fe8108-3de7-48bb-b768-117699ca6414\") " pod="openshift-monitoring/node-exporter-nlk9v" Sep 08 08:49:10.919493 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.919468 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/925bdd29-aa93-45e4-a301-e74835b7f114-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-hqrm8\" (UID: \"925bdd29-aa93-45e4-a301-e74835b7f114\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqrm8" Sep 08 08:49:10.919614 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.919598 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a2fe8108-3de7-48bb-b768-117699ca6414-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nlk9v\" (UID: \"a2fe8108-3de7-48bb-b768-117699ca6414\") " pod="openshift-monitoring/node-exporter-nlk9v" Sep 08 08:49:10.919614 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.919607 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/925bdd29-aa93-45e4-a301-e74835b7f114-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-hqrm8\" (UID: \"925bdd29-aa93-45e4-a301-e74835b7f114\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqrm8" Sep 08 08:49:10.933110 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.933086 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-schx9\" (UniqueName: \"kubernetes.io/projected/a2fe8108-3de7-48bb-b768-117699ca6414-kube-api-access-schx9\") pod \"node-exporter-nlk9v\" (UID: \"a2fe8108-3de7-48bb-b768-117699ca6414\") " pod="openshift-monitoring/node-exporter-nlk9v" Sep 08 08:49:10.933525 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.933502 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmsrk\" (UniqueName: \"kubernetes.io/projected/925bdd29-aa93-45e4-a301-e74835b7f114-kube-api-access-cmsrk\") pod \"kube-state-metrics-74cb57df7-hqrm8\" (UID: \"925bdd29-aa93-45e4-a301-e74835b7f114\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqrm8" Sep 08 08:49:10.970510 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.970460 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqrm8" Sep 08 08:49:10.997653 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:10.997630 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nlk9v" Sep 08 08:49:11.008521 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:49:11.008493 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2fe8108_3de7_48bb_b768_117699ca6414.slice/crio-317f426e2765a99909cb1f66f10d8e5a42d5d554c37dc71e917a470abd59b335 WatchSource:0}: Error finding container 317f426e2765a99909cb1f66f10d8e5a42d5d554c37dc71e917a470abd59b335: Status 404 returned error can't find the container with id 317f426e2765a99909cb1f66f10d8e5a42d5d554c37dc71e917a470abd59b335 Sep 08 08:49:11.189092 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.188895 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-zb8rb"] Sep 08 08:49:11.191531 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:49:11.191497 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb167784_18d1_42da_9020_85970a1e6fdd.slice/crio-bfa2a0174888e422611993c8f11366bbbda76c561392033f4ce923daa2ba8ae3 WatchSource:0}: Error finding container bfa2a0174888e422611993c8f11366bbbda76c561392033f4ce923daa2ba8ae3: Status 404 returned error can't find the container with id bfa2a0174888e422611993c8f11366bbbda76c561392033f4ce923daa2ba8ae3 Sep 08 08:49:11.259470 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.258892 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-hqrm8"] Sep 08 08:49:11.262266 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:49:11.262020 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod925bdd29_aa93_45e4_a301_e74835b7f114.slice/crio-1b06bbe8a43bf1621024c9f63cb007e1408453ef869a46b179f340585e45bc9f WatchSource:0}: Error finding container 1b06bbe8a43bf1621024c9f63cb007e1408453ef869a46b179f340585e45bc9f: Status 404 returned error can't find the container with id 1b06bbe8a43bf1621024c9f63cb007e1408453ef869a46b179f340585e45bc9f Sep 08 08:49:11.403993 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.403952 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-z4smd" Sep 08 08:49:11.524016 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.523967 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nlk9v" event={"ID":"a2fe8108-3de7-48bb-b768-117699ca6414","Type":"ContainerStarted","Data":"317f426e2765a99909cb1f66f10d8e5a42d5d554c37dc71e917a470abd59b335"} Sep 08 08:49:11.525717 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.525688 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zb8rb" event={"ID":"cb167784-18d1-42da-9020-85970a1e6fdd","Type":"ContainerStarted","Data":"837ccdbd6761465538106055c0e4b9e17441894cb490cfd219ce1d9fe4524270"} Sep 08 08:49:11.525834 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.525742 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zb8rb" event={"ID":"cb167784-18d1-42da-9020-85970a1e6fdd","Type":"ContainerStarted","Data":"b029c9a5d89947007c97c34fa6c51a9929df919d4353bad7a41c7f210bdb1af9"} Sep 08 08:49:11.525834 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.525778 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zb8rb" event={"ID":"cb167784-18d1-42da-9020-85970a1e6fdd","Type":"ContainerStarted","Data":"bfa2a0174888e422611993c8f11366bbbda76c561392033f4ce923daa2ba8ae3"} Sep 08 08:49:11.526924 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.526896 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqrm8" event={"ID":"925bdd29-aa93-45e4-a301-e74835b7f114","Type":"ContainerStarted","Data":"1b06bbe8a43bf1621024c9f63cb007e1408453ef869a46b179f340585e45bc9f"} Sep 08 08:49:11.719591 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.719546 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 08:49:11.722470 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.722450 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:11.728707 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.728477 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 08 08:49:11.728707 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.728573 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 08 08:49:11.728707 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.728477 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 08 08:49:11.729242 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.728951 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 08 08:49:11.729242 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.729000 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-bfmrl\"" Sep 08 08:49:11.729577 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.729560 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 08 08:49:11.729646 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.729622 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 08 08:49:11.729802 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.729786 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 08 08:49:11.729918 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.729905 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 08 08:49:11.730101 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.730083 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 08 08:49:11.750174 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.750150 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 08:49:11.822851 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.822778 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:11.822851 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.822835 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:11.823068 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.822919 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:11.823068 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.823015 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:11.823179 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.823071 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:11.823179 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.823152 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-config-volume\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:11.823282 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.823192 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:11.823282 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.823234 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-config-out\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:11.823282 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.823257 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:11.823437 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.823298 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbx84\" (UniqueName: \"kubernetes.io/projected/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-kube-api-access-rbx84\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:11.823437 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.823350 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-web-config\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:11.823437 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.823378 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:11.823586 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.823434 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:11.924134 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.924103 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:11.924335 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.924194 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:11.924335 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.924285 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:11.924335 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.924325 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-config-volume\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:11.924502 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.924346 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:11.924502 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.924369 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-config-out\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:11.924502 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.924398 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:11.924502 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.924430 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbx84\" (UniqueName: \"kubernetes.io/projected/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-kube-api-access-rbx84\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:11.924502 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.924455 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-web-config\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:11.924502 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.924497 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:11.924777 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.924538 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:11.924777 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.924592 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:11.924777 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.924619 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:11.925241 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.925160 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:11.925493 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.925468 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:11.927535 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:49:11.927514 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-alertmanager-trusted-ca-bundle podName:81a4c7fb-1d14-49b7-8653-2f8487aec0c2 nodeName:}" failed. No retries permitted until 2026-09-08 08:49:12.427493134 +0000 UTC m=+98.935599887 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "81a4c7fb-1d14-49b7-8653-2f8487aec0c2") : configmap references non-existent config key: ca-bundle.crt Sep 08 08:49:11.928708 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.928202 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:11.928708 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.928570 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:11.928708 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.928651 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:11.931288 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.931255 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-config-volume\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:11.931486 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.931305 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:11.931486 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.931367 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:11.931744 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.931720 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:11.931964 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.931946 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-config-out\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:11.932199 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.932183 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-web-config\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:11.939301 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:11.939281 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbx84\" (UniqueName: \"kubernetes.io/projected/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-kube-api-access-rbx84\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:12.428999 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:12.428938 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:12.429901 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:12.429873 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:12.631368 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:12.631346 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:49:12.731432 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:12.731375 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-registry-tls\") pod \"image-registry-77696969ff-xzvfk\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " pod="openshift-image-registry/image-registry-77696969ff-xzvfk" Sep 08 08:49:12.731712 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:12.731478 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/47fa42d5-9f58-489c-8031-4a366e32dfc1-metrics-tls\") pod \"dns-default-wcgdd\" (UID: \"47fa42d5-9f58-489c-8031-4a366e32dfc1\") " pod="openshift-dns/dns-default-wcgdd" Sep 08 08:49:12.734005 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:12.733961 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/47fa42d5-9f58-489c-8031-4a366e32dfc1-metrics-tls\") pod \"dns-default-wcgdd\" (UID: \"47fa42d5-9f58-489c-8031-4a366e32dfc1\") " pod="openshift-dns/dns-default-wcgdd" Sep 08 08:49:12.735169 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:12.735107 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-registry-tls\") pod \"image-registry-77696969ff-xzvfk\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " pod="openshift-image-registry/image-registry-77696969ff-xzvfk" Sep 08 08:49:12.927883 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:12.927809 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 08:49:12.930236 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:49:12.930168 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81a4c7fb_1d14_49b7_8653_2f8487aec0c2.slice/crio-163c21f4c4cb268a80b33f4ab9724f4571230e98e23a4a71a757d228d938abcc WatchSource:0}: Error finding container 163c21f4c4cb268a80b33f4ab9724f4571230e98e23a4a71a757d228d938abcc: Status 404 returned error can't find the container with id 163c21f4c4cb268a80b33f4ab9724f4571230e98e23a4a71a757d228d938abcc Sep 08 08:49:13.013707 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:13.013680 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-kl6tk\"" Sep 08 08:49:13.021894 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:13.021871 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77696969ff-xzvfk" Sep 08 08:49:13.023531 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:13.023504 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pkgjn\"" Sep 08 08:49:13.031844 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:13.031823 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wcgdd" Sep 08 08:49:13.317938 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:13.317906 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77696969ff-xzvfk"] Sep 08 08:49:13.328818 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:49:13.328788 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7af2e1e_49c0_4489_9f5b_32bb9d086ccd.slice/crio-2ca81a105d0e5720c2e7839d45d40e9b49a03c19c1b8dd3634828e7b2018b6b3 WatchSource:0}: Error finding container 2ca81a105d0e5720c2e7839d45d40e9b49a03c19c1b8dd3634828e7b2018b6b3: Status 404 returned error can't find the container with id 2ca81a105d0e5720c2e7839d45d40e9b49a03c19c1b8dd3634828e7b2018b6b3 Sep 08 08:49:13.329315 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:13.329233 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wcgdd"] Sep 08 08:49:13.331691 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:49:13.331672 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47fa42d5_9f58_489c_8031_4a366e32dfc1.slice/crio-713e1b4886122c8a66ecda27ab13389661efe4b536513391364a3c21fdabfadd WatchSource:0}: Error finding container 713e1b4886122c8a66ecda27ab13389661efe4b536513391364a3c21fdabfadd: Status 404 returned error can't find the container with id 713e1b4886122c8a66ecda27ab13389661efe4b536513391364a3c21fdabfadd Sep 08 08:49:13.539847 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:13.539798 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"81a4c7fb-1d14-49b7-8653-2f8487aec0c2","Type":"ContainerStarted","Data":"163c21f4c4cb268a80b33f4ab9724f4571230e98e23a4a71a757d228d938abcc"} Sep 08 08:49:13.544675 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:13.544640 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqrm8" event={"ID":"925bdd29-aa93-45e4-a301-e74835b7f114","Type":"ContainerStarted","Data":"0aab3fa5dfc7818d0e23429a3116b13c6e67e3f1e8d26042c11143a95c9ee48e"} Sep 08 08:49:13.544784 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:13.544704 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqrm8" event={"ID":"925bdd29-aa93-45e4-a301-e74835b7f114","Type":"ContainerStarted","Data":"3eabd5b7547c976c332071620a11967fecef2584e9cfe424c24f73cfaa74cfba"} Sep 08 08:49:13.544784 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:13.544733 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqrm8" event={"ID":"925bdd29-aa93-45e4-a301-e74835b7f114","Type":"ContainerStarted","Data":"876f294236f00a69e8fb329ccfe7943adb001f9372e37c8ef7cf7fd573c79a45"} Sep 08 08:49:13.545929 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:13.545895 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wcgdd" event={"ID":"47fa42d5-9f58-489c-8031-4a366e32dfc1","Type":"ContainerStarted","Data":"713e1b4886122c8a66ecda27ab13389661efe4b536513391364a3c21fdabfadd"} Sep 08 08:49:13.548207 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:13.548178 2554 generic.go:356] "Generic (PLEG): container finished" podID="a2fe8108-3de7-48bb-b768-117699ca6414" containerID="39143408da686ed831a59d4230a8b5f6a72a5da58ea2c05201f3f37b08c3535b" exitCode=0 Sep 08 08:49:13.548327 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:13.548282 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nlk9v" event={"ID":"a2fe8108-3de7-48bb-b768-117699ca6414","Type":"ContainerDied","Data":"39143408da686ed831a59d4230a8b5f6a72a5da58ea2c05201f3f37b08c3535b"} Sep 08 08:49:13.552872 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:13.552825 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zb8rb" event={"ID":"cb167784-18d1-42da-9020-85970a1e6fdd","Type":"ContainerStarted","Data":"9593a9f6072a2b12be1a34b5f3ae242de64dd0569a2c6091883d2c247507f3fc"} Sep 08 08:49:13.555269 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:13.555244 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77696969ff-xzvfk" event={"ID":"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd","Type":"ContainerStarted","Data":"fca1d2a9228b7028772f03f2afba72aff19aa465b9d84e64cdb4b97e67fd4ac2"} Sep 08 08:49:13.555398 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:13.555288 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77696969ff-xzvfk" event={"ID":"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd","Type":"ContainerStarted","Data":"2ca81a105d0e5720c2e7839d45d40e9b49a03c19c1b8dd3634828e7b2018b6b3"} Sep 08 08:49:13.555758 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:13.555739 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-77696969ff-xzvfk" Sep 08 08:49:13.571564 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:13.571514 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hqrm8" podStartSLOduration=2.197231045 podStartE2EDuration="3.571495479s" totalImagesPullingTime="1.374264434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 08:49:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:49:13.569630208 +0000 UTC m=+100.077736992" watchObservedRunningTime="2026-09-08 08:49:13.571495479 +0000 UTC m=+100.079602248" Sep 08 08:49:13.598730 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:13.598692 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-77696969ff-xzvfk" podStartSLOduration=99.598678393 podStartE2EDuration="1m39.598678393s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 08:47:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:49:13.596555609 +0000 UTC m=+100.104662378" watchObservedRunningTime="2026-09-08 08:49:13.598678393 +0000 UTC m=+100.106785161" Sep 08 08:49:13.632203 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:13.631573 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zb8rb" podStartSLOduration=2.32102122 podStartE2EDuration="3.631554233s" totalImagesPullingTime="1.310533013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 08:49:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:49:13.630636965 +0000 UTC m=+100.138743735" watchObservedRunningTime="2026-09-08 08:49:13.631554233 +0000 UTC m=+100.139661001" Sep 08 08:49:14.562265 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:14.562228 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nlk9v" event={"ID":"a2fe8108-3de7-48bb-b768-117699ca6414","Type":"ContainerStarted","Data":"d5388ae11110be73b56cc52b977f4bbe674289c6ed1c442e12907e964cfedd81"} Sep 08 08:49:14.562686 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:14.562296 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nlk9v" event={"ID":"a2fe8108-3de7-48bb-b768-117699ca6414","Type":"ContainerStarted","Data":"e82c527f88f4ef725228b1a6913b295832ceb6d5d89d306f0eb119319cac1d6a"} Sep 08 08:49:14.564215 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:14.564191 2554 generic.go:356] "Generic (PLEG): container finished" podID="81a4c7fb-1d14-49b7-8653-2f8487aec0c2" containerID="e383c2dde4bdf0101c42ab3e64fa7a83f09f70594f0b825d466ace3c67b31f90" exitCode=0 Sep 08 08:49:14.564358 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:14.564290 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"81a4c7fb-1d14-49b7-8653-2f8487aec0c2","Type":"ContainerDied","Data":"e383c2dde4bdf0101c42ab3e64fa7a83f09f70594f0b825d466ace3c67b31f90"} Sep 08 08:49:14.585600 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:14.585556 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-nlk9v" podStartSLOduration=2.178667839 podStartE2EDuration="4.585539242s" totalImagesPullingTime="2.344896599s" totalInitContainerRuntime="61.974804ms" isStatefulPod=true podCreationTimestamp="2026-09-08 08:49:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:49:14.584547979 +0000 UTC m=+101.092654749" watchObservedRunningTime="2026-09-08 08:49:14.585539242 +0000 UTC m=+101.093646010" Sep 08 08:49:15.323600 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:15.323569 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-f4r8r"] Sep 08 08:49:15.326051 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:15.326034 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-f4r8r" Sep 08 08:49:15.328050 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:15.328026 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Sep 08 08:49:15.328309 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:15.328295 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-2k7tq\"" Sep 08 08:49:15.334127 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:15.334109 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-f4r8r"] Sep 08 08:49:15.360966 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:15.360944 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5z6xl\" (UniqueName: \"kubernetes.io/projected/c7041779-779f-44e5-a529-85744578a714-kube-api-access-5z6xl\") pod \"monitoring-plugin-85bbbbbf4-f4r8r\" (UID: \"c7041779-779f-44e5-a529-85744578a714\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-f4r8r" Sep 08 08:49:15.361073 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:15.360999 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c7041779-779f-44e5-a529-85744578a714-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-f4r8r\" (UID: \"c7041779-779f-44e5-a529-85744578a714\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-f4r8r" Sep 08 08:49:15.462510 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:15.462474 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5z6xl\" (UniqueName: \"kubernetes.io/projected/c7041779-779f-44e5-a529-85744578a714-kube-api-access-5z6xl\") pod \"monitoring-plugin-85bbbbbf4-f4r8r\" (UID: \"c7041779-779f-44e5-a529-85744578a714\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-f4r8r" Sep 08 08:49:15.462617 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:15.462541 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c7041779-779f-44e5-a529-85744578a714-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-f4r8r\" (UID: \"c7041779-779f-44e5-a529-85744578a714\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-f4r8r" Sep 08 08:49:15.462683 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:49:15.462667 2554 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Sep 08 08:49:15.462754 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:49:15.462742 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7041779-779f-44e5-a529-85744578a714-monitoring-plugin-cert podName:c7041779-779f-44e5-a529-85744578a714 nodeName:}" failed. No retries permitted until 2026-09-08 08:49:15.962723555 +0000 UTC m=+102.470830310 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/c7041779-779f-44e5-a529-85744578a714-monitoring-plugin-cert") pod "monitoring-plugin-85bbbbbf4-f4r8r" (UID: "c7041779-779f-44e5-a529-85744578a714") : secret "monitoring-plugin-cert" not found Sep 08 08:49:15.473735 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:15.473705 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5z6xl\" (UniqueName: \"kubernetes.io/projected/c7041779-779f-44e5-a529-85744578a714-kube-api-access-5z6xl\") pod \"monitoring-plugin-85bbbbbf4-f4r8r\" (UID: \"c7041779-779f-44e5-a529-85744578a714\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-f4r8r" Sep 08 08:49:15.571078 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:15.571044 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wcgdd" event={"ID":"47fa42d5-9f58-489c-8031-4a366e32dfc1","Type":"ContainerStarted","Data":"dc4404ed1e930547ca2fc8d064f128202f10a3df4600956ca97d89d11fe99953"} Sep 08 08:49:15.571502 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:15.571111 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wcgdd" event={"ID":"47fa42d5-9f58-489c-8031-4a366e32dfc1","Type":"ContainerStarted","Data":"62bdf161264c7a8aaeb908da476faee043653345ce1a14059031c625b6bf4044"} Sep 08 08:49:15.589395 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:15.589304 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-wcgdd" podStartSLOduration=65.990240084 podStartE2EDuration="1m7.589288249s" totalImagesPullingTime="1.599048165s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 08:48:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:49:15.588264882 +0000 UTC m=+102.096371652" watchObservedRunningTime="2026-09-08 08:49:15.589288249 +0000 UTC m=+102.097395017" Sep 08 08:49:15.969394 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:15.969363 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c7041779-779f-44e5-a529-85744578a714-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-f4r8r\" (UID: \"c7041779-779f-44e5-a529-85744578a714\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-f4r8r" Sep 08 08:49:15.972336 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:15.972308 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c7041779-779f-44e5-a529-85744578a714-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-f4r8r\" (UID: \"c7041779-779f-44e5-a529-85744578a714\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-f4r8r" Sep 08 08:49:16.236178 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.236155 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-f4r8r" Sep 08 08:49:16.508762 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.508738 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-f4r8r"] Sep 08 08:49:16.512119 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:49:16.512092 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7041779_779f_44e5_a529_85744578a714.slice/crio-6cb66ee4596127fdd13822fb457e2d9280a531454c2cef4dd714c88dfdc4ec77 WatchSource:0}: Error finding container 6cb66ee4596127fdd13822fb457e2d9280a531454c2cef4dd714c88dfdc4ec77: Status 404 returned error can't find the container with id 6cb66ee4596127fdd13822fb457e2d9280a531454c2cef4dd714c88dfdc4ec77 Sep 08 08:49:16.578334 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.578304 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"81a4c7fb-1d14-49b7-8653-2f8487aec0c2","Type":"ContainerStarted","Data":"9f708009d97ec089495d2702f534b883066cd7c256cd1d95ed773d60e27178f9"} Sep 08 08:49:16.578617 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.578374 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"81a4c7fb-1d14-49b7-8653-2f8487aec0c2","Type":"ContainerStarted","Data":"27a2b4a3bf7f062de2cafd7c7059fe1df9103c436a2d59f6600160a5117d495e"} Sep 08 08:49:16.578617 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.578416 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"81a4c7fb-1d14-49b7-8653-2f8487aec0c2","Type":"ContainerStarted","Data":"22af240c75b2f4e09630bc74129f11d13a524d5b2e308009bfbd22134c1265af"} Sep 08 08:49:16.578617 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.578449 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"81a4c7fb-1d14-49b7-8653-2f8487aec0c2","Type":"ContainerStarted","Data":"2672cb42502af55f96b57c94b0ea8333b407168d4c7e4d757859690ca41fa4fd"} Sep 08 08:49:16.578617 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.578476 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"81a4c7fb-1d14-49b7-8653-2f8487aec0c2","Type":"ContainerStarted","Data":"b4e24e62dd4adc82b9f9a50df135d9637a9278bfa2751b5cf0117c2724c3335c"} Sep 08 08:49:16.579476 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.579453 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-f4r8r" event={"ID":"c7041779-779f-44e5-a529-85744578a714","Type":"ContainerStarted","Data":"6cb66ee4596127fdd13822fb457e2d9280a531454c2cef4dd714c88dfdc4ec77"} Sep 08 08:49:16.579685 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.579670 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-wcgdd" Sep 08 08:49:16.896920 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.896813 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 08 08:49:16.907187 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.907160 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:16.910932 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.910909 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 08 08:49:16.911088 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.910958 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 08 08:49:16.911088 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.911068 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 08 08:49:16.911620 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.911600 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 08 08:49:16.911708 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.911675 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 08 08:49:16.911992 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.911957 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 08 08:49:16.912069 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.912043 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 08 08:49:16.912154 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.911964 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-blm64lqjltb67\"" Sep 08 08:49:16.912346 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.912328 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 08 08:49:16.912702 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.912483 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-gmmlb\"" Sep 08 08:49:16.912702 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.912390 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 08 08:49:16.912838 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.912804 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 08 08:49:16.915596 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.915569 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 08 08:49:16.915705 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.915684 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 08 08:49:16.916193 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.916170 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 08 08:49:16.916658 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.916628 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 08 08:49:16.920694 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.918925 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 08 08:49:16.933728 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.933700 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 08 08:49:16.977327 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.977297 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:16.977456 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.977341 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:16.977456 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.977359 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:16.977520 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.977494 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:16.977573 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.977531 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/140a435b-031f-4bc2-96a2-176063bb9b3d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:16.977605 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.977595 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/140a435b-031f-4bc2-96a2-176063bb9b3d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:16.977639 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.977617 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:16.977669 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.977654 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:16.977710 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.977690 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-web-config\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:16.977761 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.977748 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lq22j\" (UniqueName: \"kubernetes.io/projected/140a435b-031f-4bc2-96a2-176063bb9b3d-kube-api-access-lq22j\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:16.977797 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.977772 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:16.977797 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.977792 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/140a435b-031f-4bc2-96a2-176063bb9b3d-config-out\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:16.977864 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.977810 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:16.977895 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.977876 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:16.977927 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.977906 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:16.977969 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.977930 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:16.978046 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.978004 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-config\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:16.978109 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.978043 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:16.978109 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.978078 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:16.978197 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:16.978106 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.079415 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.079378 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.079585 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.079434 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.079585 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.079460 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.079585 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.079503 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.079585 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.079530 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/140a435b-031f-4bc2-96a2-176063bb9b3d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.079585 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.079571 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/140a435b-031f-4bc2-96a2-176063bb9b3d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.079834 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.079597 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.080293 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.080036 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.080293 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.080082 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-web-config\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.080293 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.080120 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lq22j\" (UniqueName: \"kubernetes.io/projected/140a435b-031f-4bc2-96a2-176063bb9b3d-kube-api-access-lq22j\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.080293 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.080152 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.080293 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.080169 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.080293 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.080183 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/140a435b-031f-4bc2-96a2-176063bb9b3d-config-out\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.080293 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.080233 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.080293 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.080265 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/140a435b-031f-4bc2-96a2-176063bb9b3d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.080293 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.080292 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.080795 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.080293 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.080795 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.080352 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.080795 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.080354 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.080795 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.080382 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.080795 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.080423 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-config\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.080795 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.080449 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.080795 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.080481 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.080795 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.080507 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.081952 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.081392 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.082909 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.082887 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.083508 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.083483 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.086361 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.086321 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-config\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.086361 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.086346 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.086678 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.086638 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.087094 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.087070 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.087242 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.087214 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/140a435b-031f-4bc2-96a2-176063bb9b3d-config-out\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.087354 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.087289 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/140a435b-031f-4bc2-96a2-176063bb9b3d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.087354 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.087320 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.087461 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.087398 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.088450 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.088431 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.088808 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.088787 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-web-config\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.089076 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.089055 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.089662 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.089621 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.097656 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.097637 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lq22j\" (UniqueName: \"kubernetes.io/projected/140a435b-031f-4bc2-96a2-176063bb9b3d-kube-api-access-lq22j\") pod \"prometheus-k8s-0\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.224163 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.224131 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:17.530614 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:17.530591 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 08 08:49:17.600280 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:49:17.600248 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod140a435b_031f_4bc2_96a2_176063bb9b3d.slice/crio-4a8f3c9b9e25fab60cf6bd963a750741fb837e3930af4bfc00374b128b3a54ff WatchSource:0}: Error finding container 4a8f3c9b9e25fab60cf6bd963a750741fb837e3930af4bfc00374b128b3a54ff: Status 404 returned error can't find the container with id 4a8f3c9b9e25fab60cf6bd963a750741fb837e3930af4bfc00374b128b3a54ff Sep 08 08:49:18.590644 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:18.590572 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"81a4c7fb-1d14-49b7-8653-2f8487aec0c2","Type":"ContainerStarted","Data":"6eee8504716cc3f6b5d10b58a790514f94e68609ea48e920d9428cc8d05674c8"} Sep 08 08:49:18.592130 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:18.592110 2554 generic.go:356] "Generic (PLEG): container finished" podID="140a435b-031f-4bc2-96a2-176063bb9b3d" containerID="0fe24dc0ceab0d1aca3e3f3d62d46331c3b51bd44b9e4d1090899b50d652d6ea" exitCode=0 Sep 08 08:49:18.592214 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:18.592196 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"140a435b-031f-4bc2-96a2-176063bb9b3d","Type":"ContainerDied","Data":"0fe24dc0ceab0d1aca3e3f3d62d46331c3b51bd44b9e4d1090899b50d652d6ea"} Sep 08 08:49:18.592291 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:18.592277 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"140a435b-031f-4bc2-96a2-176063bb9b3d","Type":"ContainerStarted","Data":"4a8f3c9b9e25fab60cf6bd963a750741fb837e3930af4bfc00374b128b3a54ff"} Sep 08 08:49:18.593821 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:18.593757 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-f4r8r" event={"ID":"c7041779-779f-44e5-a529-85744578a714","Type":"ContainerStarted","Data":"efcc5ca1a4565a7a2d9f96fd9fb6e790b21f831747b472c8e996f3fc4848a908"} Sep 08 08:49:18.593966 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:18.593944 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-f4r8r" Sep 08 08:49:18.599514 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:18.599499 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-f4r8r" Sep 08 08:49:18.625571 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:18.622796 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.957340105 podStartE2EDuration="7.622777673s" totalImagesPullingTime="4.648108895s" totalInitContainerRuntime="17.328673ms" isStatefulPod=false podCreationTimestamp="2026-09-08 08:49:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:49:18.61989442 +0000 UTC m=+105.128007488" watchObservedRunningTime="2026-09-08 08:49:18.622777673 +0000 UTC m=+105.130884441" Sep 08 08:49:18.675370 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:18.675324 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-f4r8r" podStartSLOduration=1.8980242550000002 podStartE2EDuration="3.675307666s" totalImagesPullingTime="1.777283411s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 08:49:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:49:18.674372009 +0000 UTC m=+105.182478776" watchObservedRunningTime="2026-09-08 08:49:18.675307666 +0000 UTC m=+105.183414434" Sep 08 08:49:18.777212 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:18.777175 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-77696969ff-xzvfk"] Sep 08 08:49:21.027078 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:21.027042 2554 scope.go:117] "RemoveContainer" containerID="09a649711655a6771a6220122114685eaf09edd0abcdff0c196816538f2746cf" Sep 08 08:49:21.610041 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:21.610008 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xnd5g_df2da370-7d8d-414d-b82f-3d857fadd070/console-operator/2.log" Sep 08 08:49:21.610227 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:21.610134 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-xnd5g" event={"ID":"df2da370-7d8d-414d-b82f-3d857fadd070","Type":"ContainerStarted","Data":"a08a546f90313c3b1f3812d8b43a6e5c2043e2304775a0feb11c04595534bdb1"} Sep 08 08:49:21.610813 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:21.610435 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-xnd5g" Sep 08 08:49:21.638026 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:21.637998 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-xnd5g" Sep 08 08:49:21.662511 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:21.662457 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-xnd5g" podStartSLOduration=52.757987327 podStartE2EDuration="56.662434574s" totalImagesPullingTime="3.904447247s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 08:48:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:49:21.629422859 +0000 UTC m=+108.137529627" watchObservedRunningTime="2026-09-08 08:49:21.662434574 +0000 UTC m=+108.170541345" Sep 08 08:49:24.624847 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:24.624813 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"140a435b-031f-4bc2-96a2-176063bb9b3d","Type":"ContainerStarted","Data":"e9d5dd0af4b16cf638a40321a421154b253df56b983903dd6b85a05eb190da6c"} Sep 08 08:49:24.625224 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:24.624902 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"140a435b-031f-4bc2-96a2-176063bb9b3d","Type":"ContainerStarted","Data":"bc29e2817a8aeade8af3b34516144f38407d6e78be296cda978e63f59dbf63d1"} Sep 08 08:49:26.586658 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:26.586629 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-wcgdd" Sep 08 08:49:27.643926 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:27.643895 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"140a435b-031f-4bc2-96a2-176063bb9b3d","Type":"ContainerStarted","Data":"61c18b80d9c399ef85945ef84ebf0bbb6c490ee1762bac1fd4eb01ceffd0afc7"} Sep 08 08:49:27.644315 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:27.643954 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"140a435b-031f-4bc2-96a2-176063bb9b3d","Type":"ContainerStarted","Data":"69cb885649e51341e17684555d99de6d40046ff7c62ac370e953cd6f3af90229"} Sep 08 08:49:27.644315 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:27.644007 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"140a435b-031f-4bc2-96a2-176063bb9b3d","Type":"ContainerStarted","Data":"775efc4fdffe6b9dc63cd8a95578bec27b20397cc0151cc68ff00322e27016c0"} Sep 08 08:49:27.644315 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:27.644035 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"140a435b-031f-4bc2-96a2-176063bb9b3d","Type":"ContainerStarted","Data":"80d891d01bb50b68732f0b836274125b0b624dcd873662a8fb1677cfa5a22df3"} Sep 08 08:49:27.675417 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:27.675362 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.051061835 podStartE2EDuration="11.675342314s" totalImagesPullingTime="8.604720295s" totalInitContainerRuntime="19.560184ms" isStatefulPod=false podCreationTimestamp="2026-09-08 08:49:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:49:27.673069647 +0000 UTC m=+114.181176416" watchObservedRunningTime="2026-09-08 08:49:27.675342314 +0000 UTC m=+114.183449083" Sep 08 08:49:28.783488 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:28.783458 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-77696969ff-xzvfk" Sep 08 08:49:32.225457 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:32.225419 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:49:35.677558 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:35.677532 2554 generic.go:356] "Generic (PLEG): container finished" podID="bd212df8-59bb-46ae-8a18-3d1eb3cbf305" containerID="471c1c73f15d928e40e05f7ae2d5e6015fc7bbf96789b122e8cbe1cd5af06bfa" exitCode=0 Sep 08 08:49:35.677858 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:35.677606 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hs6rq" event={"ID":"bd212df8-59bb-46ae-8a18-3d1eb3cbf305","Type":"ContainerDied","Data":"471c1c73f15d928e40e05f7ae2d5e6015fc7bbf96789b122e8cbe1cd5af06bfa"} Sep 08 08:49:35.677925 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:35.677912 2554 scope.go:117] "RemoveContainer" containerID="471c1c73f15d928e40e05f7ae2d5e6015fc7bbf96789b122e8cbe1cd5af06bfa" Sep 08 08:49:36.683989 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:36.683891 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hs6rq" event={"ID":"bd212df8-59bb-46ae-8a18-3d1eb3cbf305","Type":"ContainerStarted","Data":"0cee66420a142371469f9b3e57525c3e277b5eceb671ddb2d6cc261d932076d7"} Sep 08 08:49:36.685446 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:36.685422 2554 generic.go:356] "Generic (PLEG): container finished" podID="5ee267e6-6baf-499a-9e98-78e38f849123" containerID="63275bb9b010ab620521f8d67f705e1242312ca4e64cee5e4544b2ab9ab9eaec" exitCode=0 Sep 08 08:49:36.685548 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:36.685455 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-v5vkr" event={"ID":"5ee267e6-6baf-499a-9e98-78e38f849123","Type":"ContainerDied","Data":"63275bb9b010ab620521f8d67f705e1242312ca4e64cee5e4544b2ab9ab9eaec"} Sep 08 08:49:36.685724 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:36.685711 2554 scope.go:117] "RemoveContainer" containerID="63275bb9b010ab620521f8d67f705e1242312ca4e64cee5e4544b2ab9ab9eaec" Sep 08 08:49:37.692080 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:37.692039 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-v5vkr" event={"ID":"5ee267e6-6baf-499a-9e98-78e38f849123","Type":"ContainerStarted","Data":"f25f79868c7f136c9fd37bc0c18607aaf034620e5b90077a003b571b0a107be5"} Sep 08 08:49:40.840219 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:40.840193 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_81a4c7fb-1d14-49b7-8653-2f8487aec0c2/init-config-reloader/0.log" Sep 08 08:49:41.040181 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:41.040152 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_81a4c7fb-1d14-49b7-8653-2f8487aec0c2/alertmanager/0.log" Sep 08 08:49:41.240040 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:41.240015 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_81a4c7fb-1d14-49b7-8653-2f8487aec0c2/config-reloader/0.log" Sep 08 08:49:41.440556 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:41.440526 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_81a4c7fb-1d14-49b7-8653-2f8487aec0c2/kube-rbac-proxy-web/0.log" Sep 08 08:49:41.639956 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:41.639881 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_81a4c7fb-1d14-49b7-8653-2f8487aec0c2/kube-rbac-proxy/0.log" Sep 08 08:49:41.839903 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:41.839872 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_81a4c7fb-1d14-49b7-8653-2f8487aec0c2/kube-rbac-proxy-metric/0.log" Sep 08 08:49:42.040941 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:42.040914 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_81a4c7fb-1d14-49b7-8653-2f8487aec0c2/prom-label-proxy/0.log" Sep 08 08:49:42.440876 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:42.440796 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-hqrm8_925bdd29-aa93-45e4-a301-e74835b7f114/kube-state-metrics/0.log" Sep 08 08:49:42.640170 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:42.640144 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-hqrm8_925bdd29-aa93-45e4-a301-e74835b7f114/kube-rbac-proxy-main/0.log" Sep 08 08:49:42.840385 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:42.840354 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-hqrm8_925bdd29-aa93-45e4-a301-e74835b7f114/kube-rbac-proxy-self/0.log" Sep 08 08:49:43.240071 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:43.240036 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-f4r8r_c7041779-779f-44e5-a529-85744578a714/monitoring-plugin/0.log" Sep 08 08:49:43.710837 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:43.710809 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57716ea8-fa5c-47c2-a186-7a0acbf6ae38-metrics-certs\") pod \"network-metrics-daemon-9xk7m\" (UID: \"57716ea8-fa5c-47c2-a186-7a0acbf6ae38\") " pod="openshift-multus/network-metrics-daemon-9xk7m" Sep 08 08:49:43.713093 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:43.713068 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57716ea8-fa5c-47c2-a186-7a0acbf6ae38-metrics-certs\") pod \"network-metrics-daemon-9xk7m\" (UID: \"57716ea8-fa5c-47c2-a186-7a0acbf6ae38\") " pod="openshift-multus/network-metrics-daemon-9xk7m" Sep 08 08:49:43.796323 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:43.796282 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-77696969ff-xzvfk" podUID="c7af2e1e-49c0-4489-9f5b-32bb9d086ccd" containerName="registry" containerID="cri-o://fca1d2a9228b7028772f03f2afba72aff19aa465b9d84e64cdb4b97e67fd4ac2" gracePeriod=30 Sep 08 08:49:43.938876 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:43.938847 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dt6vp\"" Sep 08 08:49:43.946912 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:43.946889 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9xk7m" Sep 08 08:49:44.039961 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.039936 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nlk9v_a2fe8108-3de7-48bb-b768-117699ca6414/init-textfile/0.log" Sep 08 08:49:44.051045 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.049145 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77696969ff-xzvfk" Sep 08 08:49:44.113341 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.113315 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-bound-sa-token\") pod \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " Sep 08 08:49:44.113443 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.113358 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-trusted-ca\") pod \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " Sep 08 08:49:44.113443 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.113405 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-ca-trust-extracted\") pod \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " Sep 08 08:49:44.113443 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.113420 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5278k\" (UniqueName: \"kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-kube-api-access-5278k\") pod \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " Sep 08 08:49:44.113576 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.113454 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-registry-tls\") pod \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " Sep 08 08:49:44.113576 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.113487 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-image-registry-private-configuration\") pod \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " Sep 08 08:49:44.113576 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.113520 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-registry-certificates\") pod \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " Sep 08 08:49:44.113576 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.113553 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-installation-pull-secrets\") pod \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\" (UID: \"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd\") " Sep 08 08:49:44.113872 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.113839 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "c7af2e1e-49c0-4489-9f5b-32bb9d086ccd" (UID: "c7af2e1e-49c0-4489-9f5b-32bb9d086ccd"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 08:49:44.114084 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.114054 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "c7af2e1e-49c0-4489-9f5b-32bb9d086ccd" (UID: "c7af2e1e-49c0-4489-9f5b-32bb9d086ccd"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 08:49:44.115757 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.115726 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c7af2e1e-49c0-4489-9f5b-32bb9d086ccd" (UID: "c7af2e1e-49c0-4489-9f5b-32bb9d086ccd"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 08:49:44.115857 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.115836 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "c7af2e1e-49c0-4489-9f5b-32bb9d086ccd" (UID: "c7af2e1e-49c0-4489-9f5b-32bb9d086ccd"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:49:44.116023 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.115993 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "c7af2e1e-49c0-4489-9f5b-32bb9d086ccd" (UID: "c7af2e1e-49c0-4489-9f5b-32bb9d086ccd"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 08:49:44.116139 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.116110 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-kube-api-access-5278k" (OuterVolumeSpecName: "kube-api-access-5278k") pod "c7af2e1e-49c0-4489-9f5b-32bb9d086ccd" (UID: "c7af2e1e-49c0-4489-9f5b-32bb9d086ccd"). InnerVolumeSpecName "kube-api-access-5278k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 08:49:44.116202 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.116148 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "c7af2e1e-49c0-4489-9f5b-32bb9d086ccd" (UID: "c7af2e1e-49c0-4489-9f5b-32bb9d086ccd"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:49:44.122136 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.122102 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "c7af2e1e-49c0-4489-9f5b-32bb9d086ccd" (UID: "c7af2e1e-49c0-4489-9f5b-32bb9d086ccd"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 08:49:44.214616 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.214568 2554 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-bound-sa-token\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:49:44.214616 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.214588 2554 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-trusted-ca\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:49:44.214616 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.214598 2554 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-ca-trust-extracted\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:49:44.214616 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.214607 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5278k\" (UniqueName: \"kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-kube-api-access-5278k\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:49:44.214616 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.214616 2554 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-registry-tls\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:49:44.214809 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.214625 2554 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-image-registry-private-configuration\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:49:44.214809 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.214634 2554 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-registry-certificates\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:49:44.214809 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.214642 2554 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd-installation-pull-secrets\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:49:44.223764 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.223746 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9xk7m"] Sep 08 08:49:44.226519 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:49:44.226496 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57716ea8_fa5c_47c2_a186_7a0acbf6ae38.slice/crio-01e0bc538b7f3cdf76667a4a29ed4896d6c65acb512598dd51462838f8225ce1 WatchSource:0}: Error finding container 01e0bc538b7f3cdf76667a4a29ed4896d6c65acb512598dd51462838f8225ce1: Status 404 returned error can't find the container with id 01e0bc538b7f3cdf76667a4a29ed4896d6c65acb512598dd51462838f8225ce1 Sep 08 08:49:44.241204 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.241181 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nlk9v_a2fe8108-3de7-48bb-b768-117699ca6414/node-exporter/0.log" Sep 08 08:49:44.440367 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.440341 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nlk9v_a2fe8108-3de7-48bb-b768-117699ca6414/kube-rbac-proxy/0.log" Sep 08 08:49:44.720690 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.720662 2554 generic.go:356] "Generic (PLEG): container finished" podID="c7af2e1e-49c0-4489-9f5b-32bb9d086ccd" containerID="fca1d2a9228b7028772f03f2afba72aff19aa465b9d84e64cdb4b97e67fd4ac2" exitCode=0 Sep 08 08:49:44.720813 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.720709 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77696969ff-xzvfk" Sep 08 08:49:44.720813 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.720758 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77696969ff-xzvfk" event={"ID":"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd","Type":"ContainerDied","Data":"fca1d2a9228b7028772f03f2afba72aff19aa465b9d84e64cdb4b97e67fd4ac2"} Sep 08 08:49:44.720918 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.720818 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77696969ff-xzvfk" event={"ID":"c7af2e1e-49c0-4489-9f5b-32bb9d086ccd","Type":"ContainerDied","Data":"2ca81a105d0e5720c2e7839d45d40e9b49a03c19c1b8dd3634828e7b2018b6b3"} Sep 08 08:49:44.720918 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.720843 2554 scope.go:117] "RemoveContainer" containerID="fca1d2a9228b7028772f03f2afba72aff19aa465b9d84e64cdb4b97e67fd4ac2" Sep 08 08:49:44.722510 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.722471 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9xk7m" event={"ID":"57716ea8-fa5c-47c2-a186-7a0acbf6ae38","Type":"ContainerStarted","Data":"01e0bc538b7f3cdf76667a4a29ed4896d6c65acb512598dd51462838f8225ce1"} Sep 08 08:49:44.729940 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.729923 2554 scope.go:117] "RemoveContainer" containerID="fca1d2a9228b7028772f03f2afba72aff19aa465b9d84e64cdb4b97e67fd4ac2" Sep 08 08:49:44.730249 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:49:44.730216 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fca1d2a9228b7028772f03f2afba72aff19aa465b9d84e64cdb4b97e67fd4ac2\": container with ID starting with fca1d2a9228b7028772f03f2afba72aff19aa465b9d84e64cdb4b97e67fd4ac2 not found: ID does not exist" containerID="fca1d2a9228b7028772f03f2afba72aff19aa465b9d84e64cdb4b97e67fd4ac2" Sep 08 08:49:44.730323 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.730247 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fca1d2a9228b7028772f03f2afba72aff19aa465b9d84e64cdb4b97e67fd4ac2"} err="failed to get container status \"fca1d2a9228b7028772f03f2afba72aff19aa465b9d84e64cdb4b97e67fd4ac2\": rpc error: code = NotFound desc = could not find container \"fca1d2a9228b7028772f03f2afba72aff19aa465b9d84e64cdb4b97e67fd4ac2\": container with ID starting with fca1d2a9228b7028772f03f2afba72aff19aa465b9d84e64cdb4b97e67fd4ac2 not found: ID does not exist" Sep 08 08:49:44.743245 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.743226 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-77696969ff-xzvfk"] Sep 08 08:49:44.749684 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:44.749666 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-77696969ff-xzvfk"] Sep 08 08:49:45.240548 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:45.240520 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-zb8rb_cb167784-18d1-42da-9020-85970a1e6fdd/kube-rbac-proxy-main/0.log" Sep 08 08:49:45.440225 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:45.440198 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-zb8rb_cb167784-18d1-42da-9020-85970a1e6fdd/kube-rbac-proxy-self/0.log" Sep 08 08:49:45.641543 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:45.641462 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-zb8rb_cb167784-18d1-42da-9020-85970a1e6fdd/openshift-state-metrics/0.log" Sep 08 08:49:45.845949 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:45.845919 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_140a435b-031f-4bc2-96a2-176063bb9b3d/init-config-reloader/0.log" Sep 08 08:49:46.034013 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:46.033968 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7af2e1e-49c0-4489-9f5b-32bb9d086ccd" path="/var/lib/kubelet/pods/c7af2e1e-49c0-4489-9f5b-32bb9d086ccd/volumes" Sep 08 08:49:46.043436 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:46.043411 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_140a435b-031f-4bc2-96a2-176063bb9b3d/prometheus/0.log" Sep 08 08:49:46.241441 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:46.241405 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_140a435b-031f-4bc2-96a2-176063bb9b3d/config-reloader/0.log" Sep 08 08:49:46.441693 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:46.441616 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_140a435b-031f-4bc2-96a2-176063bb9b3d/thanos-sidecar/0.log" Sep 08 08:49:46.641202 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:46.641173 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_140a435b-031f-4bc2-96a2-176063bb9b3d/kube-rbac-proxy-web/0.log" Sep 08 08:49:46.843395 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:46.843371 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_140a435b-031f-4bc2-96a2-176063bb9b3d/kube-rbac-proxy/0.log" Sep 08 08:49:47.042259 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:47.042166 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_140a435b-031f-4bc2-96a2-176063bb9b3d/kube-rbac-proxy-thanos/0.log" Sep 08 08:49:47.243330 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:47.243300 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-slgg5_1aa1093f-f3ec-4854-81f6-a23ce51ff061/prometheus-operator/0.log" Sep 08 08:49:47.440753 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:47.440668 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-slgg5_1aa1093f-f3ec-4854-81f6-a23ce51ff061/kube-rbac-proxy/0.log" Sep 08 08:49:47.639670 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:47.639645 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-gpzzn_caf19724-9691-48b9-8c53-c5c908cd8bdc/prometheus-operator-admission-webhook/0.log" Sep 08 08:49:47.737880 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:47.737847 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9xk7m" event={"ID":"57716ea8-fa5c-47c2-a186-7a0acbf6ae38","Type":"ContainerStarted","Data":"93585bddcad33a26cffadb58b7b88656ae896f86d6f25748c543251711d7faf0"} Sep 08 08:49:47.738086 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:47.737904 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9xk7m" event={"ID":"57716ea8-fa5c-47c2-a186-7a0acbf6ae38","Type":"ContainerStarted","Data":"567fcfacb5883605fdb796898d0fa3c068144ae42c5c3c70212f56c9c6493d22"} Sep 08 08:49:47.755361 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:47.755309 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9xk7m" podStartSLOduration=131.193940856 podStartE2EDuration="2m13.755291455s" totalImagesPullingTime="2.561350599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 08:47:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:49:47.752563232 +0000 UTC m=+134.260670000" watchObservedRunningTime="2026-09-08 08:49:47.755291455 +0000 UTC m=+134.263398225" Sep 08 08:49:49.042022 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:49.041947 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-kcwgm_242ea579-b683-4fe7-a860-89724bce185b/networking-console-plugin/0.log" Sep 08 08:49:49.241064 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:49.241030 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xnd5g_df2da370-7d8d-414d-b82f-3d857fadd070/console-operator/2.log" Sep 08 08:49:49.443218 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:49.443135 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xnd5g_df2da370-7d8d-414d-b82f-3d857fadd070/console-operator/3.log" Sep 08 08:49:50.440707 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:49:50.440671 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lp6dm_923c000a-89fd-4a8f-a8ff-5aa9a32c3509/serve-healthcheck-canary/0.log" Sep 08 08:50:17.224888 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:17.224854 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:17.245054 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:17.245030 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:17.877890 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:17.877863 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:30.933101 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:30.933029 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 08:50:30.933600 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:30.933348 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="81a4c7fb-1d14-49b7-8653-2f8487aec0c2" containerName="alertmanager" containerID="cri-o://b4e24e62dd4adc82b9f9a50df135d9637a9278bfa2751b5cf0117c2724c3335c" gracePeriod=120 Sep 08 08:50:30.933600 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:30.933370 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="81a4c7fb-1d14-49b7-8653-2f8487aec0c2" containerName="prom-label-proxy" containerID="cri-o://6eee8504716cc3f6b5d10b58a790514f94e68609ea48e920d9428cc8d05674c8" gracePeriod=120 Sep 08 08:50:30.933600 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:30.933393 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="81a4c7fb-1d14-49b7-8653-2f8487aec0c2" containerName="kube-rbac-proxy" containerID="cri-o://27a2b4a3bf7f062de2cafd7c7059fe1df9103c436a2d59f6600160a5117d495e" gracePeriod=120 Sep 08 08:50:30.933600 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:30.933423 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="81a4c7fb-1d14-49b7-8653-2f8487aec0c2" containerName="config-reloader" containerID="cri-o://2672cb42502af55f96b57c94b0ea8333b407168d4c7e4d757859690ca41fa4fd" gracePeriod=120 Sep 08 08:50:30.933600 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:30.933421 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="81a4c7fb-1d14-49b7-8653-2f8487aec0c2" containerName="kube-rbac-proxy-metric" containerID="cri-o://9f708009d97ec089495d2702f534b883066cd7c256cd1d95ed773d60e27178f9" gracePeriod=120 Sep 08 08:50:30.933600 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:30.933420 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="81a4c7fb-1d14-49b7-8653-2f8487aec0c2" containerName="kube-rbac-proxy-web" containerID="cri-o://22af240c75b2f4e09630bc74129f11d13a524d5b2e308009bfbd22134c1265af" gracePeriod=120 Sep 08 08:50:31.929087 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:31.929054 2554 generic.go:356] "Generic (PLEG): container finished" podID="81a4c7fb-1d14-49b7-8653-2f8487aec0c2" containerID="6eee8504716cc3f6b5d10b58a790514f94e68609ea48e920d9428cc8d05674c8" exitCode=0 Sep 08 08:50:31.929087 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:31.929080 2554 generic.go:356] "Generic (PLEG): container finished" podID="81a4c7fb-1d14-49b7-8653-2f8487aec0c2" containerID="27a2b4a3bf7f062de2cafd7c7059fe1df9103c436a2d59f6600160a5117d495e" exitCode=0 Sep 08 08:50:31.929087 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:31.929086 2554 generic.go:356] "Generic (PLEG): container finished" podID="81a4c7fb-1d14-49b7-8653-2f8487aec0c2" containerID="2672cb42502af55f96b57c94b0ea8333b407168d4c7e4d757859690ca41fa4fd" exitCode=0 Sep 08 08:50:31.929087 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:31.929092 2554 generic.go:356] "Generic (PLEG): container finished" podID="81a4c7fb-1d14-49b7-8653-2f8487aec0c2" containerID="b4e24e62dd4adc82b9f9a50df135d9637a9278bfa2751b5cf0117c2724c3335c" exitCode=0 Sep 08 08:50:31.929348 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:31.929121 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"81a4c7fb-1d14-49b7-8653-2f8487aec0c2","Type":"ContainerDied","Data":"6eee8504716cc3f6b5d10b58a790514f94e68609ea48e920d9428cc8d05674c8"} Sep 08 08:50:31.929348 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:31.929205 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"81a4c7fb-1d14-49b7-8653-2f8487aec0c2","Type":"ContainerDied","Data":"27a2b4a3bf7f062de2cafd7c7059fe1df9103c436a2d59f6600160a5117d495e"} Sep 08 08:50:31.929348 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:31.929232 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"81a4c7fb-1d14-49b7-8653-2f8487aec0c2","Type":"ContainerDied","Data":"2672cb42502af55f96b57c94b0ea8333b407168d4c7e4d757859690ca41fa4fd"} Sep 08 08:50:31.929348 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:31.929257 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"81a4c7fb-1d14-49b7-8653-2f8487aec0c2","Type":"ContainerDied","Data":"b4e24e62dd4adc82b9f9a50df135d9637a9278bfa2751b5cf0117c2724c3335c"} Sep 08 08:50:32.172311 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.172290 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:32.282018 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.281963 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-cluster-tls-config\") pod \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " Sep 08 08:50:32.282018 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.282015 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbx84\" (UniqueName: \"kubernetes.io/projected/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-kube-api-access-rbx84\") pod \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " Sep 08 08:50:32.282258 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.282053 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-alertmanager-trusted-ca-bundle\") pod \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " Sep 08 08:50:32.282258 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.282076 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-metrics-client-ca\") pod \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " Sep 08 08:50:32.282258 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.282099 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-config-volume\") pod \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " Sep 08 08:50:32.282258 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.282122 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-web-config\") pod \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " Sep 08 08:50:32.282258 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.282149 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-tls-assets\") pod \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " Sep 08 08:50:32.282258 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.282183 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-config-out\") pod \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " Sep 08 08:50:32.282258 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.282213 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-secret-alertmanager-kube-rbac-proxy-web\") pod \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " Sep 08 08:50:32.282258 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.282251 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " Sep 08 08:50:32.282673 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.282277 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-alertmanager-main-db\") pod \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " Sep 08 08:50:32.282673 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.282304 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-secret-alertmanager-main-tls\") pod \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " Sep 08 08:50:32.282673 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.282341 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-secret-alertmanager-kube-rbac-proxy\") pod \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\" (UID: \"81a4c7fb-1d14-49b7-8653-2f8487aec0c2\") " Sep 08 08:50:32.282673 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.282470 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "81a4c7fb-1d14-49b7-8653-2f8487aec0c2" (UID: "81a4c7fb-1d14-49b7-8653-2f8487aec0c2"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 08:50:32.282673 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.282483 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "81a4c7fb-1d14-49b7-8653-2f8487aec0c2" (UID: "81a4c7fb-1d14-49b7-8653-2f8487aec0c2"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 08:50:32.282673 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.282639 2554 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:50:32.282673 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.282662 2554 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-metrics-client-ca\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:50:32.283120 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.283044 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "81a4c7fb-1d14-49b7-8653-2f8487aec0c2" (UID: "81a4c7fb-1d14-49b7-8653-2f8487aec0c2"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 08:50:32.285274 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.285234 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-config-volume" (OuterVolumeSpecName: "config-volume") pod "81a4c7fb-1d14-49b7-8653-2f8487aec0c2" (UID: "81a4c7fb-1d14-49b7-8653-2f8487aec0c2"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:50:32.285405 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.285367 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-config-out" (OuterVolumeSpecName: "config-out") pod "81a4c7fb-1d14-49b7-8653-2f8487aec0c2" (UID: "81a4c7fb-1d14-49b7-8653-2f8487aec0c2"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 08:50:32.285476 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.285432 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-kube-api-access-rbx84" (OuterVolumeSpecName: "kube-api-access-rbx84") pod "81a4c7fb-1d14-49b7-8653-2f8487aec0c2" (UID: "81a4c7fb-1d14-49b7-8653-2f8487aec0c2"). InnerVolumeSpecName "kube-api-access-rbx84". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 08:50:32.285476 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.285444 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "81a4c7fb-1d14-49b7-8653-2f8487aec0c2" (UID: "81a4c7fb-1d14-49b7-8653-2f8487aec0c2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:50:32.285476 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.285462 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "81a4c7fb-1d14-49b7-8653-2f8487aec0c2" (UID: "81a4c7fb-1d14-49b7-8653-2f8487aec0c2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:50:32.285642 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.285479 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "81a4c7fb-1d14-49b7-8653-2f8487aec0c2" (UID: "81a4c7fb-1d14-49b7-8653-2f8487aec0c2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:50:32.285881 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.285844 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "81a4c7fb-1d14-49b7-8653-2f8487aec0c2" (UID: "81a4c7fb-1d14-49b7-8653-2f8487aec0c2"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:50:32.286754 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.286732 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "81a4c7fb-1d14-49b7-8653-2f8487aec0c2" (UID: "81a4c7fb-1d14-49b7-8653-2f8487aec0c2"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 08:50:32.289169 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.289143 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "81a4c7fb-1d14-49b7-8653-2f8487aec0c2" (UID: "81a4c7fb-1d14-49b7-8653-2f8487aec0c2"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:50:32.295896 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.295873 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-web-config" (OuterVolumeSpecName: "web-config") pod "81a4c7fb-1d14-49b7-8653-2f8487aec0c2" (UID: "81a4c7fb-1d14-49b7-8653-2f8487aec0c2"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:50:32.383641 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.383618 2554 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-cluster-tls-config\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:50:32.383641 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.383638 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbx84\" (UniqueName: \"kubernetes.io/projected/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-kube-api-access-rbx84\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:50:32.383750 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.383650 2554 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-config-volume\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:50:32.383750 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.383657 2554 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-web-config\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:50:32.383750 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.383665 2554 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-tls-assets\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:50:32.383750 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.383672 2554 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-config-out\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:50:32.383750 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.383681 2554 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:50:32.383750 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.383690 2554 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:50:32.383750 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.383699 2554 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-alertmanager-main-db\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:50:32.383750 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.383708 2554 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-secret-alertmanager-main-tls\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:50:32.383750 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.383716 2554 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/81a4c7fb-1d14-49b7-8653-2f8487aec0c2-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:50:32.936586 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.936541 2554 generic.go:356] "Generic (PLEG): container finished" podID="81a4c7fb-1d14-49b7-8653-2f8487aec0c2" containerID="9f708009d97ec089495d2702f534b883066cd7c256cd1d95ed773d60e27178f9" exitCode=0 Sep 08 08:50:32.936586 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.936572 2554 generic.go:356] "Generic (PLEG): container finished" podID="81a4c7fb-1d14-49b7-8653-2f8487aec0c2" containerID="22af240c75b2f4e09630bc74129f11d13a524d5b2e308009bfbd22134c1265af" exitCode=0 Sep 08 08:50:32.936768 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.936625 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:32.936768 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.936622 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"81a4c7fb-1d14-49b7-8653-2f8487aec0c2","Type":"ContainerDied","Data":"9f708009d97ec089495d2702f534b883066cd7c256cd1d95ed773d60e27178f9"} Sep 08 08:50:32.936883 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.936773 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"81a4c7fb-1d14-49b7-8653-2f8487aec0c2","Type":"ContainerDied","Data":"22af240c75b2f4e09630bc74129f11d13a524d5b2e308009bfbd22134c1265af"} Sep 08 08:50:32.936883 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.936816 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"81a4c7fb-1d14-49b7-8653-2f8487aec0c2","Type":"ContainerDied","Data":"163c21f4c4cb268a80b33f4ab9724f4571230e98e23a4a71a757d228d938abcc"} Sep 08 08:50:32.936883 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.936840 2554 scope.go:117] "RemoveContainer" containerID="6eee8504716cc3f6b5d10b58a790514f94e68609ea48e920d9428cc8d05674c8" Sep 08 08:50:32.945340 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.945207 2554 scope.go:117] "RemoveContainer" containerID="9f708009d97ec089495d2702f534b883066cd7c256cd1d95ed773d60e27178f9" Sep 08 08:50:32.952601 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.952586 2554 scope.go:117] "RemoveContainer" containerID="27a2b4a3bf7f062de2cafd7c7059fe1df9103c436a2d59f6600160a5117d495e" Sep 08 08:50:32.959659 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.959636 2554 scope.go:117] "RemoveContainer" containerID="22af240c75b2f4e09630bc74129f11d13a524d5b2e308009bfbd22134c1265af" Sep 08 08:50:32.960089 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.960068 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 08:50:32.965064 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.965042 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 08:50:32.967509 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.967492 2554 scope.go:117] "RemoveContainer" containerID="2672cb42502af55f96b57c94b0ea8333b407168d4c7e4d757859690ca41fa4fd" Sep 08 08:50:32.974349 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.974332 2554 scope.go:117] "RemoveContainer" containerID="b4e24e62dd4adc82b9f9a50df135d9637a9278bfa2751b5cf0117c2724c3335c" Sep 08 08:50:32.981138 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.981117 2554 scope.go:117] "RemoveContainer" containerID="e383c2dde4bdf0101c42ab3e64fa7a83f09f70594f0b825d466ace3c67b31f90" Sep 08 08:50:32.988142 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.988126 2554 scope.go:117] "RemoveContainer" containerID="6eee8504716cc3f6b5d10b58a790514f94e68609ea48e920d9428cc8d05674c8" Sep 08 08:50:32.988415 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:50:32.988398 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6eee8504716cc3f6b5d10b58a790514f94e68609ea48e920d9428cc8d05674c8\": container with ID starting with 6eee8504716cc3f6b5d10b58a790514f94e68609ea48e920d9428cc8d05674c8 not found: ID does not exist" containerID="6eee8504716cc3f6b5d10b58a790514f94e68609ea48e920d9428cc8d05674c8" Sep 08 08:50:32.988471 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.988422 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6eee8504716cc3f6b5d10b58a790514f94e68609ea48e920d9428cc8d05674c8"} err="failed to get container status \"6eee8504716cc3f6b5d10b58a790514f94e68609ea48e920d9428cc8d05674c8\": rpc error: code = NotFound desc = could not find container \"6eee8504716cc3f6b5d10b58a790514f94e68609ea48e920d9428cc8d05674c8\": container with ID starting with 6eee8504716cc3f6b5d10b58a790514f94e68609ea48e920d9428cc8d05674c8 not found: ID does not exist" Sep 08 08:50:32.988471 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.988439 2554 scope.go:117] "RemoveContainer" containerID="9f708009d97ec089495d2702f534b883066cd7c256cd1d95ed773d60e27178f9" Sep 08 08:50:32.988660 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:50:32.988642 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f708009d97ec089495d2702f534b883066cd7c256cd1d95ed773d60e27178f9\": container with ID starting with 9f708009d97ec089495d2702f534b883066cd7c256cd1d95ed773d60e27178f9 not found: ID does not exist" containerID="9f708009d97ec089495d2702f534b883066cd7c256cd1d95ed773d60e27178f9" Sep 08 08:50:32.988728 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.988671 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f708009d97ec089495d2702f534b883066cd7c256cd1d95ed773d60e27178f9"} err="failed to get container status \"9f708009d97ec089495d2702f534b883066cd7c256cd1d95ed773d60e27178f9\": rpc error: code = NotFound desc = could not find container \"9f708009d97ec089495d2702f534b883066cd7c256cd1d95ed773d60e27178f9\": container with ID starting with 9f708009d97ec089495d2702f534b883066cd7c256cd1d95ed773d60e27178f9 not found: ID does not exist" Sep 08 08:50:32.988728 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.988693 2554 scope.go:117] "RemoveContainer" containerID="27a2b4a3bf7f062de2cafd7c7059fe1df9103c436a2d59f6600160a5117d495e" Sep 08 08:50:32.988968 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:50:32.988950 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27a2b4a3bf7f062de2cafd7c7059fe1df9103c436a2d59f6600160a5117d495e\": container with ID starting with 27a2b4a3bf7f062de2cafd7c7059fe1df9103c436a2d59f6600160a5117d495e not found: ID does not exist" containerID="27a2b4a3bf7f062de2cafd7c7059fe1df9103c436a2d59f6600160a5117d495e" Sep 08 08:50:32.989039 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.988989 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27a2b4a3bf7f062de2cafd7c7059fe1df9103c436a2d59f6600160a5117d495e"} err="failed to get container status \"27a2b4a3bf7f062de2cafd7c7059fe1df9103c436a2d59f6600160a5117d495e\": rpc error: code = NotFound desc = could not find container \"27a2b4a3bf7f062de2cafd7c7059fe1df9103c436a2d59f6600160a5117d495e\": container with ID starting with 27a2b4a3bf7f062de2cafd7c7059fe1df9103c436a2d59f6600160a5117d495e not found: ID does not exist" Sep 08 08:50:32.989039 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.989008 2554 scope.go:117] "RemoveContainer" containerID="22af240c75b2f4e09630bc74129f11d13a524d5b2e308009bfbd22134c1265af" Sep 08 08:50:32.989246 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:50:32.989230 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22af240c75b2f4e09630bc74129f11d13a524d5b2e308009bfbd22134c1265af\": container with ID starting with 22af240c75b2f4e09630bc74129f11d13a524d5b2e308009bfbd22134c1265af not found: ID does not exist" containerID="22af240c75b2f4e09630bc74129f11d13a524d5b2e308009bfbd22134c1265af" Sep 08 08:50:32.989304 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.989252 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22af240c75b2f4e09630bc74129f11d13a524d5b2e308009bfbd22134c1265af"} err="failed to get container status \"22af240c75b2f4e09630bc74129f11d13a524d5b2e308009bfbd22134c1265af\": rpc error: code = NotFound desc = could not find container \"22af240c75b2f4e09630bc74129f11d13a524d5b2e308009bfbd22134c1265af\": container with ID starting with 22af240c75b2f4e09630bc74129f11d13a524d5b2e308009bfbd22134c1265af not found: ID does not exist" Sep 08 08:50:32.989304 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.989273 2554 scope.go:117] "RemoveContainer" containerID="2672cb42502af55f96b57c94b0ea8333b407168d4c7e4d757859690ca41fa4fd" Sep 08 08:50:32.989507 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:50:32.989491 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2672cb42502af55f96b57c94b0ea8333b407168d4c7e4d757859690ca41fa4fd\": container with ID starting with 2672cb42502af55f96b57c94b0ea8333b407168d4c7e4d757859690ca41fa4fd not found: ID does not exist" containerID="2672cb42502af55f96b57c94b0ea8333b407168d4c7e4d757859690ca41fa4fd" Sep 08 08:50:32.989542 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.989512 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2672cb42502af55f96b57c94b0ea8333b407168d4c7e4d757859690ca41fa4fd"} err="failed to get container status \"2672cb42502af55f96b57c94b0ea8333b407168d4c7e4d757859690ca41fa4fd\": rpc error: code = NotFound desc = could not find container \"2672cb42502af55f96b57c94b0ea8333b407168d4c7e4d757859690ca41fa4fd\": container with ID starting with 2672cb42502af55f96b57c94b0ea8333b407168d4c7e4d757859690ca41fa4fd not found: ID does not exist" Sep 08 08:50:32.989542 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.989525 2554 scope.go:117] "RemoveContainer" containerID="b4e24e62dd4adc82b9f9a50df135d9637a9278bfa2751b5cf0117c2724c3335c" Sep 08 08:50:32.989727 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:50:32.989710 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4e24e62dd4adc82b9f9a50df135d9637a9278bfa2751b5cf0117c2724c3335c\": container with ID starting with b4e24e62dd4adc82b9f9a50df135d9637a9278bfa2751b5cf0117c2724c3335c not found: ID does not exist" containerID="b4e24e62dd4adc82b9f9a50df135d9637a9278bfa2751b5cf0117c2724c3335c" Sep 08 08:50:32.989795 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.989732 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4e24e62dd4adc82b9f9a50df135d9637a9278bfa2751b5cf0117c2724c3335c"} err="failed to get container status \"b4e24e62dd4adc82b9f9a50df135d9637a9278bfa2751b5cf0117c2724c3335c\": rpc error: code = NotFound desc = could not find container \"b4e24e62dd4adc82b9f9a50df135d9637a9278bfa2751b5cf0117c2724c3335c\": container with ID starting with b4e24e62dd4adc82b9f9a50df135d9637a9278bfa2751b5cf0117c2724c3335c not found: ID does not exist" Sep 08 08:50:32.989795 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.989752 2554 scope.go:117] "RemoveContainer" containerID="e383c2dde4bdf0101c42ab3e64fa7a83f09f70594f0b825d466ace3c67b31f90" Sep 08 08:50:32.989961 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:50:32.989946 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e383c2dde4bdf0101c42ab3e64fa7a83f09f70594f0b825d466ace3c67b31f90\": container with ID starting with e383c2dde4bdf0101c42ab3e64fa7a83f09f70594f0b825d466ace3c67b31f90 not found: ID does not exist" containerID="e383c2dde4bdf0101c42ab3e64fa7a83f09f70594f0b825d466ace3c67b31f90" Sep 08 08:50:32.990031 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.989964 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e383c2dde4bdf0101c42ab3e64fa7a83f09f70594f0b825d466ace3c67b31f90"} err="failed to get container status \"e383c2dde4bdf0101c42ab3e64fa7a83f09f70594f0b825d466ace3c67b31f90\": rpc error: code = NotFound desc = could not find container \"e383c2dde4bdf0101c42ab3e64fa7a83f09f70594f0b825d466ace3c67b31f90\": container with ID starting with e383c2dde4bdf0101c42ab3e64fa7a83f09f70594f0b825d466ace3c67b31f90 not found: ID does not exist" Sep 08 08:50:32.990031 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.989990 2554 scope.go:117] "RemoveContainer" containerID="6eee8504716cc3f6b5d10b58a790514f94e68609ea48e920d9428cc8d05674c8" Sep 08 08:50:32.990220 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.990192 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6eee8504716cc3f6b5d10b58a790514f94e68609ea48e920d9428cc8d05674c8"} err="failed to get container status \"6eee8504716cc3f6b5d10b58a790514f94e68609ea48e920d9428cc8d05674c8\": rpc error: code = NotFound desc = could not find container \"6eee8504716cc3f6b5d10b58a790514f94e68609ea48e920d9428cc8d05674c8\": container with ID starting with 6eee8504716cc3f6b5d10b58a790514f94e68609ea48e920d9428cc8d05674c8 not found: ID does not exist" Sep 08 08:50:32.990287 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.990221 2554 scope.go:117] "RemoveContainer" containerID="9f708009d97ec089495d2702f534b883066cd7c256cd1d95ed773d60e27178f9" Sep 08 08:50:32.990425 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.990408 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f708009d97ec089495d2702f534b883066cd7c256cd1d95ed773d60e27178f9"} err="failed to get container status \"9f708009d97ec089495d2702f534b883066cd7c256cd1d95ed773d60e27178f9\": rpc error: code = NotFound desc = could not find container \"9f708009d97ec089495d2702f534b883066cd7c256cd1d95ed773d60e27178f9\": container with ID starting with 9f708009d97ec089495d2702f534b883066cd7c256cd1d95ed773d60e27178f9 not found: ID does not exist" Sep 08 08:50:32.990470 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.990425 2554 scope.go:117] "RemoveContainer" containerID="27a2b4a3bf7f062de2cafd7c7059fe1df9103c436a2d59f6600160a5117d495e" Sep 08 08:50:32.990629 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.990613 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27a2b4a3bf7f062de2cafd7c7059fe1df9103c436a2d59f6600160a5117d495e"} err="failed to get container status \"27a2b4a3bf7f062de2cafd7c7059fe1df9103c436a2d59f6600160a5117d495e\": rpc error: code = NotFound desc = could not find container \"27a2b4a3bf7f062de2cafd7c7059fe1df9103c436a2d59f6600160a5117d495e\": container with ID starting with 27a2b4a3bf7f062de2cafd7c7059fe1df9103c436a2d59f6600160a5117d495e not found: ID does not exist" Sep 08 08:50:32.990695 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.990631 2554 scope.go:117] "RemoveContainer" containerID="22af240c75b2f4e09630bc74129f11d13a524d5b2e308009bfbd22134c1265af" Sep 08 08:50:32.990857 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.990839 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22af240c75b2f4e09630bc74129f11d13a524d5b2e308009bfbd22134c1265af"} err="failed to get container status \"22af240c75b2f4e09630bc74129f11d13a524d5b2e308009bfbd22134c1265af\": rpc error: code = NotFound desc = could not find container \"22af240c75b2f4e09630bc74129f11d13a524d5b2e308009bfbd22134c1265af\": container with ID starting with 22af240c75b2f4e09630bc74129f11d13a524d5b2e308009bfbd22134c1265af not found: ID does not exist" Sep 08 08:50:32.990903 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.990858 2554 scope.go:117] "RemoveContainer" containerID="2672cb42502af55f96b57c94b0ea8333b407168d4c7e4d757859690ca41fa4fd" Sep 08 08:50:32.991101 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.991081 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2672cb42502af55f96b57c94b0ea8333b407168d4c7e4d757859690ca41fa4fd"} err="failed to get container status \"2672cb42502af55f96b57c94b0ea8333b407168d4c7e4d757859690ca41fa4fd\": rpc error: code = NotFound desc = could not find container \"2672cb42502af55f96b57c94b0ea8333b407168d4c7e4d757859690ca41fa4fd\": container with ID starting with 2672cb42502af55f96b57c94b0ea8333b407168d4c7e4d757859690ca41fa4fd not found: ID does not exist" Sep 08 08:50:32.991151 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.991101 2554 scope.go:117] "RemoveContainer" containerID="b4e24e62dd4adc82b9f9a50df135d9637a9278bfa2751b5cf0117c2724c3335c" Sep 08 08:50:32.991388 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.991371 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4e24e62dd4adc82b9f9a50df135d9637a9278bfa2751b5cf0117c2724c3335c"} err="failed to get container status \"b4e24e62dd4adc82b9f9a50df135d9637a9278bfa2751b5cf0117c2724c3335c\": rpc error: code = NotFound desc = could not find container \"b4e24e62dd4adc82b9f9a50df135d9637a9278bfa2751b5cf0117c2724c3335c\": container with ID starting with b4e24e62dd4adc82b9f9a50df135d9637a9278bfa2751b5cf0117c2724c3335c not found: ID does not exist" Sep 08 08:50:32.991388 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.991387 2554 scope.go:117] "RemoveContainer" containerID="e383c2dde4bdf0101c42ab3e64fa7a83f09f70594f0b825d466ace3c67b31f90" Sep 08 08:50:32.991601 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.991587 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e383c2dde4bdf0101c42ab3e64fa7a83f09f70594f0b825d466ace3c67b31f90"} err="failed to get container status \"e383c2dde4bdf0101c42ab3e64fa7a83f09f70594f0b825d466ace3c67b31f90\": rpc error: code = NotFound desc = could not find container \"e383c2dde4bdf0101c42ab3e64fa7a83f09f70594f0b825d466ace3c67b31f90\": container with ID starting with e383c2dde4bdf0101c42ab3e64fa7a83f09f70594f0b825d466ace3c67b31f90 not found: ID does not exist" Sep 08 08:50:32.994531 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.994509 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 08:50:32.994819 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.994807 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81a4c7fb-1d14-49b7-8653-2f8487aec0c2" containerName="kube-rbac-proxy-metric" Sep 08 08:50:32.994857 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.994821 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="81a4c7fb-1d14-49b7-8653-2f8487aec0c2" containerName="kube-rbac-proxy-metric" Sep 08 08:50:32.994857 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.994835 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81a4c7fb-1d14-49b7-8653-2f8487aec0c2" containerName="kube-rbac-proxy" Sep 08 08:50:32.994857 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.994843 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="81a4c7fb-1d14-49b7-8653-2f8487aec0c2" containerName="kube-rbac-proxy" Sep 08 08:50:32.994857 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.994851 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81a4c7fb-1d14-49b7-8653-2f8487aec0c2" containerName="init-config-reloader" Sep 08 08:50:32.994857 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.994856 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="81a4c7fb-1d14-49b7-8653-2f8487aec0c2" containerName="init-config-reloader" Sep 08 08:50:32.995025 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.994869 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81a4c7fb-1d14-49b7-8653-2f8487aec0c2" containerName="config-reloader" Sep 08 08:50:32.995025 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.994875 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="81a4c7fb-1d14-49b7-8653-2f8487aec0c2" containerName="config-reloader" Sep 08 08:50:32.995025 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.994882 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81a4c7fb-1d14-49b7-8653-2f8487aec0c2" containerName="prom-label-proxy" Sep 08 08:50:32.995025 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.994886 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="81a4c7fb-1d14-49b7-8653-2f8487aec0c2" containerName="prom-label-proxy" Sep 08 08:50:32.995025 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.994895 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81a4c7fb-1d14-49b7-8653-2f8487aec0c2" containerName="alertmanager" Sep 08 08:50:32.995025 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.994899 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="81a4c7fb-1d14-49b7-8653-2f8487aec0c2" containerName="alertmanager" Sep 08 08:50:32.995025 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.994906 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7af2e1e-49c0-4489-9f5b-32bb9d086ccd" containerName="registry" Sep 08 08:50:32.995025 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.994914 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7af2e1e-49c0-4489-9f5b-32bb9d086ccd" containerName="registry" Sep 08 08:50:32.995025 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.994922 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81a4c7fb-1d14-49b7-8653-2f8487aec0c2" containerName="kube-rbac-proxy-web" Sep 08 08:50:32.995025 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.994929 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="81a4c7fb-1d14-49b7-8653-2f8487aec0c2" containerName="kube-rbac-proxy-web" Sep 08 08:50:32.995303 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.995060 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7af2e1e-49c0-4489-9f5b-32bb9d086ccd" containerName="registry" Sep 08 08:50:32.995303 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.995072 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="81a4c7fb-1d14-49b7-8653-2f8487aec0c2" containerName="alertmanager" Sep 08 08:50:32.995303 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.995079 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="81a4c7fb-1d14-49b7-8653-2f8487aec0c2" containerName="config-reloader" Sep 08 08:50:32.995303 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.995084 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="81a4c7fb-1d14-49b7-8653-2f8487aec0c2" containerName="prom-label-proxy" Sep 08 08:50:32.995303 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.995091 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="81a4c7fb-1d14-49b7-8653-2f8487aec0c2" containerName="kube-rbac-proxy-metric" Sep 08 08:50:32.995303 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.995096 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="81a4c7fb-1d14-49b7-8653-2f8487aec0c2" containerName="kube-rbac-proxy-web" Sep 08 08:50:32.995303 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.995101 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="81a4c7fb-1d14-49b7-8653-2f8487aec0c2" containerName="kube-rbac-proxy" Sep 08 08:50:32.998872 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:32.998858 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.001415 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.001398 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 08 08:50:33.001589 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.001565 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-bfmrl\"" Sep 08 08:50:33.001697 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.001631 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 08 08:50:33.002073 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.001940 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 08 08:50:33.002073 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.001995 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 08 08:50:33.002073 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.001998 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 08 08:50:33.002299 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.001947 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 08 08:50:33.002345 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.002313 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 08 08:50:33.002345 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.002328 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 08 08:50:33.006729 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.006710 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 08 08:50:33.012601 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.012574 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 08:50:33.087090 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.087068 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d8618cc7-6b6a-4145-b71c-595ec40f4feb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.087214 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.087094 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zskfn\" (UniqueName: \"kubernetes.io/projected/d8618cc7-6b6a-4145-b71c-595ec40f4feb-kube-api-access-zskfn\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.087214 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.087123 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d8618cc7-6b6a-4145-b71c-595ec40f4feb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.087214 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.087139 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d8618cc7-6b6a-4145-b71c-595ec40f4feb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.087334 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.087213 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d8618cc7-6b6a-4145-b71c-595ec40f4feb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.087334 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.087248 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d8618cc7-6b6a-4145-b71c-595ec40f4feb-web-config\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.087334 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.087294 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d8618cc7-6b6a-4145-b71c-595ec40f4feb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.087449 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.087347 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d8618cc7-6b6a-4145-b71c-595ec40f4feb-config-volume\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.087449 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.087363 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d8618cc7-6b6a-4145-b71c-595ec40f4feb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.087449 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.087417 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8618cc7-6b6a-4145-b71c-595ec40f4feb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.087449 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.087439 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d8618cc7-6b6a-4145-b71c-595ec40f4feb-config-out\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.087587 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.087460 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d8618cc7-6b6a-4145-b71c-595ec40f4feb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.087587 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.087488 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d8618cc7-6b6a-4145-b71c-595ec40f4feb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.187837 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.187770 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d8618cc7-6b6a-4145-b71c-595ec40f4feb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.187837 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.187806 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d8618cc7-6b6a-4145-b71c-595ec40f4feb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.187837 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.187826 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d8618cc7-6b6a-4145-b71c-595ec40f4feb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.189743 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.187843 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d8618cc7-6b6a-4145-b71c-595ec40f4feb-web-config\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.189743 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.187876 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d8618cc7-6b6a-4145-b71c-595ec40f4feb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.189743 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.187911 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d8618cc7-6b6a-4145-b71c-595ec40f4feb-config-volume\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.189743 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.187927 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d8618cc7-6b6a-4145-b71c-595ec40f4feb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.189743 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.187956 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8618cc7-6b6a-4145-b71c-595ec40f4feb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.189743 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.187998 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d8618cc7-6b6a-4145-b71c-595ec40f4feb-config-out\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.189743 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.188019 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d8618cc7-6b6a-4145-b71c-595ec40f4feb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.189743 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.188049 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d8618cc7-6b6a-4145-b71c-595ec40f4feb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.189743 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.188072 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d8618cc7-6b6a-4145-b71c-595ec40f4feb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.189743 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.188092 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zskfn\" (UniqueName: \"kubernetes.io/projected/d8618cc7-6b6a-4145-b71c-595ec40f4feb-kube-api-access-zskfn\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.189743 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.188394 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d8618cc7-6b6a-4145-b71c-595ec40f4feb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.189743 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.188635 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d8618cc7-6b6a-4145-b71c-595ec40f4feb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.189743 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.189198 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8618cc7-6b6a-4145-b71c-595ec40f4feb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.191673 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.191181 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d8618cc7-6b6a-4145-b71c-595ec40f4feb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.191673 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.191394 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d8618cc7-6b6a-4145-b71c-595ec40f4feb-web-config\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.191673 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.191638 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d8618cc7-6b6a-4145-b71c-595ec40f4feb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.191673 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.191667 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d8618cc7-6b6a-4145-b71c-595ec40f4feb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.192325 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.192302 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d8618cc7-6b6a-4145-b71c-595ec40f4feb-config-volume\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.192418 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.192364 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d8618cc7-6b6a-4145-b71c-595ec40f4feb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.192867 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.192846 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d8618cc7-6b6a-4145-b71c-595ec40f4feb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.193002 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.192906 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d8618cc7-6b6a-4145-b71c-595ec40f4feb-config-out\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.193068 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.193024 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d8618cc7-6b6a-4145-b71c-595ec40f4feb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.197749 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.197729 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zskfn\" (UniqueName: \"kubernetes.io/projected/d8618cc7-6b6a-4145-b71c-595ec40f4feb-kube-api-access-zskfn\") pod \"alertmanager-main-0\" (UID: \"d8618cc7-6b6a-4145-b71c-595ec40f4feb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.308394 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.308369 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 08 08:50:33.589658 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.589634 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 08:50:33.591511 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:50:33.591483 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8618cc7_6b6a_4145_b71c_595ec40f4feb.slice/crio-db25084975fefd8ca49c9beb1e4561a95e7887e93a01d8169191e3c3f7a61f15 WatchSource:0}: Error finding container db25084975fefd8ca49c9beb1e4561a95e7887e93a01d8169191e3c3f7a61f15: Status 404 returned error can't find the container with id db25084975fefd8ca49c9beb1e4561a95e7887e93a01d8169191e3c3f7a61f15 Sep 08 08:50:33.942148 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.942115 2554 generic.go:356] "Generic (PLEG): container finished" podID="d8618cc7-6b6a-4145-b71c-595ec40f4feb" containerID="06b3c02ebea70d2bcf8531330af321ffc1802882d24427f5f606f4f33610d012" exitCode=0 Sep 08 08:50:33.942272 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.942189 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d8618cc7-6b6a-4145-b71c-595ec40f4feb","Type":"ContainerDied","Data":"06b3c02ebea70d2bcf8531330af321ffc1802882d24427f5f606f4f33610d012"} Sep 08 08:50:33.942366 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:33.942279 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d8618cc7-6b6a-4145-b71c-595ec40f4feb","Type":"ContainerStarted","Data":"db25084975fefd8ca49c9beb1e4561a95e7887e93a01d8169191e3c3f7a61f15"} Sep 08 08:50:34.035008 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:34.034661 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81a4c7fb-1d14-49b7-8653-2f8487aec0c2" path="/var/lib/kubelet/pods/81a4c7fb-1d14-49b7-8653-2f8487aec0c2/volumes" Sep 08 08:50:34.949531 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:34.949492 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-76497f5b48-g9v24"] Sep 08 08:50:34.952584 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:34.952554 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d8618cc7-6b6a-4145-b71c-595ec40f4feb","Type":"ContainerStarted","Data":"2a504ea2bea93ec5164dfa2318fb999ae2d4dbba33288505256dec587f52de05"} Sep 08 08:50:34.952721 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:34.952614 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d8618cc7-6b6a-4145-b71c-595ec40f4feb","Type":"ContainerStarted","Data":"1cca7da80533e7a3e8bd5e177b0bdeecfef0968fac625571e3435b1d19f3f0a0"} Sep 08 08:50:34.952721 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:34.952652 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d8618cc7-6b6a-4145-b71c-595ec40f4feb","Type":"ContainerStarted","Data":"9645af10d0fcf53e8da462cd5f738d6f854bf57695309fb3a38113a2fee6bd74"} Sep 08 08:50:34.952721 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:34.952688 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d8618cc7-6b6a-4145-b71c-595ec40f4feb","Type":"ContainerStarted","Data":"c338512d6232d19855d736c2eb21f2fa4e5fd168ea1369daef521d435dee1588"} Sep 08 08:50:34.952884 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:34.952724 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d8618cc7-6b6a-4145-b71c-595ec40f4feb","Type":"ContainerStarted","Data":"6e3a84f6f9a8e426cba895fc2bbe6833ef7d2bfb8eb1b455820fb28716dc0584"} Sep 08 08:50:34.952884 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:34.952763 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d8618cc7-6b6a-4145-b71c-595ec40f4feb","Type":"ContainerStarted","Data":"3c4fbeaba65816a916164ad57a0c15fff0ebf9eaac5e5d91d4662d29e8f910d9"} Sep 08 08:50:34.952884 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:34.952874 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-76497f5b48-g9v24" Sep 08 08:50:34.955160 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:34.955140 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 08 08:50:34.955272 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:34.955226 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 08 08:50:34.955272 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:34.955253 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 08 08:50:34.955272 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:34.955265 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 08 08:50:34.955391 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:34.955346 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 08 08:50:34.956132 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:34.956113 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-wt9k8\"" Sep 08 08:50:34.961877 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:34.961856 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 08 08:50:34.966232 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:34.966165 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-76497f5b48-g9v24"] Sep 08 08:50:34.985645 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:34.985591 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.972378994 podStartE2EDuration="2.985576418s" totalImagesPullingTime="0s" totalInitContainerRuntime="13.197424ms" isStatefulPod=false podCreationTimestamp="2026-09-08 08:50:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:50:34.98423455 +0000 UTC m=+181.492341333" watchObservedRunningTime="2026-09-08 08:50:34.985576418 +0000 UTC m=+181.493683185" Sep 08 08:50:35.004749 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.004722 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9f8s\" (UniqueName: \"kubernetes.io/projected/9afa9c3f-a12a-4e54-867d-0ff38b8eafa8-kube-api-access-t9f8s\") pod \"telemeter-client-76497f5b48-g9v24\" (UID: \"9afa9c3f-a12a-4e54-867d-0ff38b8eafa8\") " pod="openshift-monitoring/telemeter-client-76497f5b48-g9v24" Sep 08 08:50:35.004887 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.004780 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9afa9c3f-a12a-4e54-867d-0ff38b8eafa8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-76497f5b48-g9v24\" (UID: \"9afa9c3f-a12a-4e54-867d-0ff38b8eafa8\") " pod="openshift-monitoring/telemeter-client-76497f5b48-g9v24" Sep 08 08:50:35.004887 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.004811 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9afa9c3f-a12a-4e54-867d-0ff38b8eafa8-secret-telemeter-client\") pod \"telemeter-client-76497f5b48-g9v24\" (UID: \"9afa9c3f-a12a-4e54-867d-0ff38b8eafa8\") " pod="openshift-monitoring/telemeter-client-76497f5b48-g9v24" Sep 08 08:50:35.005060 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.004948 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9afa9c3f-a12a-4e54-867d-0ff38b8eafa8-metrics-client-ca\") pod \"telemeter-client-76497f5b48-g9v24\" (UID: \"9afa9c3f-a12a-4e54-867d-0ff38b8eafa8\") " pod="openshift-monitoring/telemeter-client-76497f5b48-g9v24" Sep 08 08:50:35.005060 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.005034 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9afa9c3f-a12a-4e54-867d-0ff38b8eafa8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-76497f5b48-g9v24\" (UID: \"9afa9c3f-a12a-4e54-867d-0ff38b8eafa8\") " pod="openshift-monitoring/telemeter-client-76497f5b48-g9v24" Sep 08 08:50:35.005251 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.005233 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9afa9c3f-a12a-4e54-867d-0ff38b8eafa8-serving-certs-ca-bundle\") pod \"telemeter-client-76497f5b48-g9v24\" (UID: \"9afa9c3f-a12a-4e54-867d-0ff38b8eafa8\") " pod="openshift-monitoring/telemeter-client-76497f5b48-g9v24" Sep 08 08:50:35.005343 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.005263 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9afa9c3f-a12a-4e54-867d-0ff38b8eafa8-federate-client-tls\") pod \"telemeter-client-76497f5b48-g9v24\" (UID: \"9afa9c3f-a12a-4e54-867d-0ff38b8eafa8\") " pod="openshift-monitoring/telemeter-client-76497f5b48-g9v24" Sep 08 08:50:35.005343 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.005327 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9afa9c3f-a12a-4e54-867d-0ff38b8eafa8-telemeter-client-tls\") pod \"telemeter-client-76497f5b48-g9v24\" (UID: \"9afa9c3f-a12a-4e54-867d-0ff38b8eafa8\") " pod="openshift-monitoring/telemeter-client-76497f5b48-g9v24" Sep 08 08:50:35.106152 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.106119 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9afa9c3f-a12a-4e54-867d-0ff38b8eafa8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-76497f5b48-g9v24\" (UID: \"9afa9c3f-a12a-4e54-867d-0ff38b8eafa8\") " pod="openshift-monitoring/telemeter-client-76497f5b48-g9v24" Sep 08 08:50:35.106152 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.106155 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9afa9c3f-a12a-4e54-867d-0ff38b8eafa8-secret-telemeter-client\") pod \"telemeter-client-76497f5b48-g9v24\" (UID: \"9afa9c3f-a12a-4e54-867d-0ff38b8eafa8\") " pod="openshift-monitoring/telemeter-client-76497f5b48-g9v24" Sep 08 08:50:35.106391 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.106174 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9afa9c3f-a12a-4e54-867d-0ff38b8eafa8-metrics-client-ca\") pod \"telemeter-client-76497f5b48-g9v24\" (UID: \"9afa9c3f-a12a-4e54-867d-0ff38b8eafa8\") " pod="openshift-monitoring/telemeter-client-76497f5b48-g9v24" Sep 08 08:50:35.106391 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.106335 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9afa9c3f-a12a-4e54-867d-0ff38b8eafa8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-76497f5b48-g9v24\" (UID: \"9afa9c3f-a12a-4e54-867d-0ff38b8eafa8\") " pod="openshift-monitoring/telemeter-client-76497f5b48-g9v24" Sep 08 08:50:35.106694 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.106631 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9afa9c3f-a12a-4e54-867d-0ff38b8eafa8-serving-certs-ca-bundle\") pod \"telemeter-client-76497f5b48-g9v24\" (UID: \"9afa9c3f-a12a-4e54-867d-0ff38b8eafa8\") " pod="openshift-monitoring/telemeter-client-76497f5b48-g9v24" Sep 08 08:50:35.106922 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.106665 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9afa9c3f-a12a-4e54-867d-0ff38b8eafa8-federate-client-tls\") pod \"telemeter-client-76497f5b48-g9v24\" (UID: \"9afa9c3f-a12a-4e54-867d-0ff38b8eafa8\") " pod="openshift-monitoring/telemeter-client-76497f5b48-g9v24" Sep 08 08:50:35.106922 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.106884 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9afa9c3f-a12a-4e54-867d-0ff38b8eafa8-telemeter-client-tls\") pod \"telemeter-client-76497f5b48-g9v24\" (UID: \"9afa9c3f-a12a-4e54-867d-0ff38b8eafa8\") " pod="openshift-monitoring/telemeter-client-76497f5b48-g9v24" Sep 08 08:50:35.107470 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.107426 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9afa9c3f-a12a-4e54-867d-0ff38b8eafa8-metrics-client-ca\") pod \"telemeter-client-76497f5b48-g9v24\" (UID: \"9afa9c3f-a12a-4e54-867d-0ff38b8eafa8\") " pod="openshift-monitoring/telemeter-client-76497f5b48-g9v24" Sep 08 08:50:35.107959 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.107568 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9f8s\" (UniqueName: \"kubernetes.io/projected/9afa9c3f-a12a-4e54-867d-0ff38b8eafa8-kube-api-access-t9f8s\") pod \"telemeter-client-76497f5b48-g9v24\" (UID: \"9afa9c3f-a12a-4e54-867d-0ff38b8eafa8\") " pod="openshift-monitoring/telemeter-client-76497f5b48-g9v24" Sep 08 08:50:35.107959 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.107863 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9afa9c3f-a12a-4e54-867d-0ff38b8eafa8-serving-certs-ca-bundle\") pod \"telemeter-client-76497f5b48-g9v24\" (UID: \"9afa9c3f-a12a-4e54-867d-0ff38b8eafa8\") " pod="openshift-monitoring/telemeter-client-76497f5b48-g9v24" Sep 08 08:50:35.108411 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.108374 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9afa9c3f-a12a-4e54-867d-0ff38b8eafa8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-76497f5b48-g9v24\" (UID: \"9afa9c3f-a12a-4e54-867d-0ff38b8eafa8\") " pod="openshift-monitoring/telemeter-client-76497f5b48-g9v24" Sep 08 08:50:35.109203 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.109176 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9afa9c3f-a12a-4e54-867d-0ff38b8eafa8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-76497f5b48-g9v24\" (UID: \"9afa9c3f-a12a-4e54-867d-0ff38b8eafa8\") " pod="openshift-monitoring/telemeter-client-76497f5b48-g9v24" Sep 08 08:50:35.114552 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.114524 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9afa9c3f-a12a-4e54-867d-0ff38b8eafa8-federate-client-tls\") pod \"telemeter-client-76497f5b48-g9v24\" (UID: \"9afa9c3f-a12a-4e54-867d-0ff38b8eafa8\") " pod="openshift-monitoring/telemeter-client-76497f5b48-g9v24" Sep 08 08:50:35.115888 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.115863 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9afa9c3f-a12a-4e54-867d-0ff38b8eafa8-telemeter-client-tls\") pod \"telemeter-client-76497f5b48-g9v24\" (UID: \"9afa9c3f-a12a-4e54-867d-0ff38b8eafa8\") " pod="openshift-monitoring/telemeter-client-76497f5b48-g9v24" Sep 08 08:50:35.116664 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.116642 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9afa9c3f-a12a-4e54-867d-0ff38b8eafa8-secret-telemeter-client\") pod \"telemeter-client-76497f5b48-g9v24\" (UID: \"9afa9c3f-a12a-4e54-867d-0ff38b8eafa8\") " pod="openshift-monitoring/telemeter-client-76497f5b48-g9v24" Sep 08 08:50:35.117880 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.117851 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9f8s\" (UniqueName: \"kubernetes.io/projected/9afa9c3f-a12a-4e54-867d-0ff38b8eafa8-kube-api-access-t9f8s\") pod \"telemeter-client-76497f5b48-g9v24\" (UID: \"9afa9c3f-a12a-4e54-867d-0ff38b8eafa8\") " pod="openshift-monitoring/telemeter-client-76497f5b48-g9v24" Sep 08 08:50:35.264800 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.264705 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 08 08:50:35.265127 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.265079 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="140a435b-031f-4bc2-96a2-176063bb9b3d" containerName="prometheus" containerID="cri-o://bc29e2817a8aeade8af3b34516144f38407d6e78be296cda978e63f59dbf63d1" gracePeriod=600 Sep 08 08:50:35.265246 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.265136 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="140a435b-031f-4bc2-96a2-176063bb9b3d" containerName="kube-rbac-proxy" containerID="cri-o://69cb885649e51341e17684555d99de6d40046ff7c62ac370e953cd6f3af90229" gracePeriod=600 Sep 08 08:50:35.265246 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.265133 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="140a435b-031f-4bc2-96a2-176063bb9b3d" containerName="kube-rbac-proxy-web" containerID="cri-o://775efc4fdffe6b9dc63cd8a95578bec27b20397cc0151cc68ff00322e27016c0" gracePeriod=600 Sep 08 08:50:35.265246 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.265090 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="140a435b-031f-4bc2-96a2-176063bb9b3d" containerName="kube-rbac-proxy-thanos" containerID="cri-o://61c18b80d9c399ef85945ef84ebf0bbb6c490ee1762bac1fd4eb01ceffd0afc7" gracePeriod=600 Sep 08 08:50:35.265246 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.265136 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="140a435b-031f-4bc2-96a2-176063bb9b3d" containerName="thanos-sidecar" containerID="cri-o://80d891d01bb50b68732f0b836274125b0b624dcd873662a8fb1677cfa5a22df3" gracePeriod=600 Sep 08 08:50:35.266194 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.265522 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="140a435b-031f-4bc2-96a2-176063bb9b3d" containerName="config-reloader" containerID="cri-o://e9d5dd0af4b16cf638a40321a421154b253df56b983903dd6b85a05eb190da6c" gracePeriod=600 Sep 08 08:50:35.266194 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.265725 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-76497f5b48-g9v24" Sep 08 08:50:35.507307 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.506939 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:35.563189 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.563161 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-76497f5b48-g9v24"] Sep 08 08:50:35.565594 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:50:35.565568 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9afa9c3f_a12a_4e54_867d_0ff38b8eafa8.slice/crio-d7e9138dfe8d863a494368fd1fcd9971a50b569ac96f39d147a5468b0d9890af WatchSource:0}: Error finding container d7e9138dfe8d863a494368fd1fcd9971a50b569ac96f39d147a5468b0d9890af: Status 404 returned error can't find the container with id d7e9138dfe8d863a494368fd1fcd9971a50b569ac96f39d147a5468b0d9890af Sep 08 08:50:35.611785 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.611760 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-configmap-metrics-client-ca\") pod \"140a435b-031f-4bc2-96a2-176063bb9b3d\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " Sep 08 08:50:35.611944 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.611796 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-prometheus-trusted-ca-bundle\") pod \"140a435b-031f-4bc2-96a2-176063bb9b3d\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " Sep 08 08:50:35.611944 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.611823 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-configmap-kubelet-serving-ca-bundle\") pod \"140a435b-031f-4bc2-96a2-176063bb9b3d\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " Sep 08 08:50:35.612086 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.612016 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-secret-grpc-tls\") pod \"140a435b-031f-4bc2-96a2-176063bb9b3d\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " Sep 08 08:50:35.612086 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.612062 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/140a435b-031f-4bc2-96a2-176063bb9b3d-prometheus-k8s-db\") pod \"140a435b-031f-4bc2-96a2-176063bb9b3d\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " Sep 08 08:50:35.612191 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.612093 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-secret-metrics-client-certs\") pod \"140a435b-031f-4bc2-96a2-176063bb9b3d\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " Sep 08 08:50:35.612191 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.612138 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/140a435b-031f-4bc2-96a2-176063bb9b3d-tls-assets\") pod \"140a435b-031f-4bc2-96a2-176063bb9b3d\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " Sep 08 08:50:35.612191 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.612148 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "140a435b-031f-4bc2-96a2-176063bb9b3d" (UID: "140a435b-031f-4bc2-96a2-176063bb9b3d"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 08:50:35.612191 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.612165 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-prometheus-k8s-rulefiles-2\") pod \"140a435b-031f-4bc2-96a2-176063bb9b3d\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " Sep 08 08:50:35.612191 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.612188 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-config\") pod \"140a435b-031f-4bc2-96a2-176063bb9b3d\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " Sep 08 08:50:35.612434 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.612188 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "140a435b-031f-4bc2-96a2-176063bb9b3d" (UID: "140a435b-031f-4bc2-96a2-176063bb9b3d"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 08:50:35.612434 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.612219 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-thanos-prometheus-http-client-file\") pod \"140a435b-031f-4bc2-96a2-176063bb9b3d\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " Sep 08 08:50:35.612434 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.612257 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lq22j\" (UniqueName: \"kubernetes.io/projected/140a435b-031f-4bc2-96a2-176063bb9b3d-kube-api-access-lq22j\") pod \"140a435b-031f-4bc2-96a2-176063bb9b3d\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " Sep 08 08:50:35.612434 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.612290 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-prometheus-k8s-rulefiles-0\") pod \"140a435b-031f-4bc2-96a2-176063bb9b3d\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " Sep 08 08:50:35.612434 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.612316 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-secret-prometheus-k8s-tls\") pod \"140a435b-031f-4bc2-96a2-176063bb9b3d\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " Sep 08 08:50:35.612434 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.612362 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"140a435b-031f-4bc2-96a2-176063bb9b3d\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " Sep 08 08:50:35.612434 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.612367 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "140a435b-031f-4bc2-96a2-176063bb9b3d" (UID: "140a435b-031f-4bc2-96a2-176063bb9b3d"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 08:50:35.612434 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.612397 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-prometheus-k8s-rulefiles-1\") pod \"140a435b-031f-4bc2-96a2-176063bb9b3d\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " Sep 08 08:50:35.612825 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.612450 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-web-config\") pod \"140a435b-031f-4bc2-96a2-176063bb9b3d\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " Sep 08 08:50:35.612825 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.612486 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"140a435b-031f-4bc2-96a2-176063bb9b3d\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " Sep 08 08:50:35.612825 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.612513 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/140a435b-031f-4bc2-96a2-176063bb9b3d-config-out\") pod \"140a435b-031f-4bc2-96a2-176063bb9b3d\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " Sep 08 08:50:35.612825 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.612543 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-configmap-serving-certs-ca-bundle\") pod \"140a435b-031f-4bc2-96a2-176063bb9b3d\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " Sep 08 08:50:35.612825 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.612588 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-secret-kube-rbac-proxy\") pod \"140a435b-031f-4bc2-96a2-176063bb9b3d\" (UID: \"140a435b-031f-4bc2-96a2-176063bb9b3d\") " Sep 08 08:50:35.612825 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.612618 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "140a435b-031f-4bc2-96a2-176063bb9b3d" (UID: "140a435b-031f-4bc2-96a2-176063bb9b3d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 08:50:35.613153 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.612844 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "140a435b-031f-4bc2-96a2-176063bb9b3d" (UID: "140a435b-031f-4bc2-96a2-176063bb9b3d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 08:50:35.613153 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.612892 2554 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:50:35.613153 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.612911 2554 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-configmap-metrics-client-ca\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:50:35.613153 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.612926 2554 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-prometheus-trusted-ca-bundle\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:50:35.613153 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.612939 2554 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:50:35.613399 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.613231 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/140a435b-031f-4bc2-96a2-176063bb9b3d-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "140a435b-031f-4bc2-96a2-176063bb9b3d" (UID: "140a435b-031f-4bc2-96a2-176063bb9b3d"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 08:50:35.614048 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.613777 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "140a435b-031f-4bc2-96a2-176063bb9b3d" (UID: "140a435b-031f-4bc2-96a2-176063bb9b3d"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 08:50:35.614585 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.614540 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "140a435b-031f-4bc2-96a2-176063bb9b3d" (UID: "140a435b-031f-4bc2-96a2-176063bb9b3d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 08:50:35.615254 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.615227 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/140a435b-031f-4bc2-96a2-176063bb9b3d-kube-api-access-lq22j" (OuterVolumeSpecName: "kube-api-access-lq22j") pod "140a435b-031f-4bc2-96a2-176063bb9b3d" (UID: "140a435b-031f-4bc2-96a2-176063bb9b3d"). InnerVolumeSpecName "kube-api-access-lq22j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 08:50:35.615514 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.615464 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/140a435b-031f-4bc2-96a2-176063bb9b3d-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "140a435b-031f-4bc2-96a2-176063bb9b3d" (UID: "140a435b-031f-4bc2-96a2-176063bb9b3d"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 08:50:35.615773 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.615748 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "140a435b-031f-4bc2-96a2-176063bb9b3d" (UID: "140a435b-031f-4bc2-96a2-176063bb9b3d"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:50:35.615956 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.615931 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "140a435b-031f-4bc2-96a2-176063bb9b3d" (UID: "140a435b-031f-4bc2-96a2-176063bb9b3d"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:50:35.616414 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.616391 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "140a435b-031f-4bc2-96a2-176063bb9b3d" (UID: "140a435b-031f-4bc2-96a2-176063bb9b3d"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:50:35.616510 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.616427 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/140a435b-031f-4bc2-96a2-176063bb9b3d-config-out" (OuterVolumeSpecName: "config-out") pod "140a435b-031f-4bc2-96a2-176063bb9b3d" (UID: "140a435b-031f-4bc2-96a2-176063bb9b3d"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 08:50:35.616735 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.616710 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "140a435b-031f-4bc2-96a2-176063bb9b3d" (UID: "140a435b-031f-4bc2-96a2-176063bb9b3d"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:50:35.617024 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.617000 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-config" (OuterVolumeSpecName: "config") pod "140a435b-031f-4bc2-96a2-176063bb9b3d" (UID: "140a435b-031f-4bc2-96a2-176063bb9b3d"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:50:35.617213 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.617191 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "140a435b-031f-4bc2-96a2-176063bb9b3d" (UID: "140a435b-031f-4bc2-96a2-176063bb9b3d"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:50:35.617370 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.617341 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "140a435b-031f-4bc2-96a2-176063bb9b3d" (UID: "140a435b-031f-4bc2-96a2-176063bb9b3d"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:50:35.617452 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.617370 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "140a435b-031f-4bc2-96a2-176063bb9b3d" (UID: "140a435b-031f-4bc2-96a2-176063bb9b3d"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:50:35.626677 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.626656 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-web-config" (OuterVolumeSpecName: "web-config") pod "140a435b-031f-4bc2-96a2-176063bb9b3d" (UID: "140a435b-031f-4bc2-96a2-176063bb9b3d"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 08:50:35.713965 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.713942 2554 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:50:35.713965 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.713967 2554 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:50:35.714104 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.713996 2554 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-web-config\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:50:35.714104 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.714007 2554 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:50:35.714104 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.714016 2554 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/140a435b-031f-4bc2-96a2-176063bb9b3d-config-out\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:50:35.714104 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.714025 2554 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:50:35.714104 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.714033 2554 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-secret-kube-rbac-proxy\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:50:35.714104 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.714041 2554 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-secret-grpc-tls\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:50:35.714104 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.714049 2554 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/140a435b-031f-4bc2-96a2-176063bb9b3d-prometheus-k8s-db\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:50:35.714104 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.714057 2554 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-secret-metrics-client-certs\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:50:35.714104 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.714065 2554 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/140a435b-031f-4bc2-96a2-176063bb9b3d-tls-assets\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:50:35.714104 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.714073 2554 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-config\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:50:35.714104 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.714080 2554 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-thanos-prometheus-http-client-file\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:50:35.714104 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.714088 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lq22j\" (UniqueName: \"kubernetes.io/projected/140a435b-031f-4bc2-96a2-176063bb9b3d-kube-api-access-lq22j\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:50:35.714104 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.714098 2554 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/140a435b-031f-4bc2-96a2-176063bb9b3d-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:50:35.714104 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.714106 2554 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/140a435b-031f-4bc2-96a2-176063bb9b3d-secret-prometheus-k8s-tls\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 08:50:35.959322 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.959294 2554 generic.go:356] "Generic (PLEG): container finished" podID="140a435b-031f-4bc2-96a2-176063bb9b3d" containerID="61c18b80d9c399ef85945ef84ebf0bbb6c490ee1762bac1fd4eb01ceffd0afc7" exitCode=0 Sep 08 08:50:35.959322 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.959317 2554 generic.go:356] "Generic (PLEG): container finished" podID="140a435b-031f-4bc2-96a2-176063bb9b3d" containerID="69cb885649e51341e17684555d99de6d40046ff7c62ac370e953cd6f3af90229" exitCode=0 Sep 08 08:50:35.959322 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.959324 2554 generic.go:356] "Generic (PLEG): container finished" podID="140a435b-031f-4bc2-96a2-176063bb9b3d" containerID="775efc4fdffe6b9dc63cd8a95578bec27b20397cc0151cc68ff00322e27016c0" exitCode=0 Sep 08 08:50:35.959805 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.959329 2554 generic.go:356] "Generic (PLEG): container finished" podID="140a435b-031f-4bc2-96a2-176063bb9b3d" containerID="80d891d01bb50b68732f0b836274125b0b624dcd873662a8fb1677cfa5a22df3" exitCode=0 Sep 08 08:50:35.959805 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.959335 2554 generic.go:356] "Generic (PLEG): container finished" podID="140a435b-031f-4bc2-96a2-176063bb9b3d" containerID="e9d5dd0af4b16cf638a40321a421154b253df56b983903dd6b85a05eb190da6c" exitCode=0 Sep 08 08:50:35.959805 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.959340 2554 generic.go:356] "Generic (PLEG): container finished" podID="140a435b-031f-4bc2-96a2-176063bb9b3d" containerID="bc29e2817a8aeade8af3b34516144f38407d6e78be296cda978e63f59dbf63d1" exitCode=0 Sep 08 08:50:35.959805 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.959369 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:35.959805 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.959371 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"140a435b-031f-4bc2-96a2-176063bb9b3d","Type":"ContainerDied","Data":"61c18b80d9c399ef85945ef84ebf0bbb6c490ee1762bac1fd4eb01ceffd0afc7"} Sep 08 08:50:35.959805 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.959434 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"140a435b-031f-4bc2-96a2-176063bb9b3d","Type":"ContainerDied","Data":"69cb885649e51341e17684555d99de6d40046ff7c62ac370e953cd6f3af90229"} Sep 08 08:50:35.959805 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.959471 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"140a435b-031f-4bc2-96a2-176063bb9b3d","Type":"ContainerDied","Data":"775efc4fdffe6b9dc63cd8a95578bec27b20397cc0151cc68ff00322e27016c0"} Sep 08 08:50:35.959805 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.959493 2554 scope.go:117] "RemoveContainer" containerID="61c18b80d9c399ef85945ef84ebf0bbb6c490ee1762bac1fd4eb01ceffd0afc7" Sep 08 08:50:35.959805 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.959514 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"140a435b-031f-4bc2-96a2-176063bb9b3d","Type":"ContainerDied","Data":"80d891d01bb50b68732f0b836274125b0b624dcd873662a8fb1677cfa5a22df3"} Sep 08 08:50:35.959805 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.959559 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"140a435b-031f-4bc2-96a2-176063bb9b3d","Type":"ContainerDied","Data":"e9d5dd0af4b16cf638a40321a421154b253df56b983903dd6b85a05eb190da6c"} Sep 08 08:50:35.959805 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.959604 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"140a435b-031f-4bc2-96a2-176063bb9b3d","Type":"ContainerDied","Data":"bc29e2817a8aeade8af3b34516144f38407d6e78be296cda978e63f59dbf63d1"} Sep 08 08:50:35.959805 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.959645 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"140a435b-031f-4bc2-96a2-176063bb9b3d","Type":"ContainerDied","Data":"4a8f3c9b9e25fab60cf6bd963a750741fb837e3930af4bfc00374b128b3a54ff"} Sep 08 08:50:35.960840 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.960815 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-76497f5b48-g9v24" event={"ID":"9afa9c3f-a12a-4e54-867d-0ff38b8eafa8","Type":"ContainerStarted","Data":"d7e9138dfe8d863a494368fd1fcd9971a50b569ac96f39d147a5468b0d9890af"} Sep 08 08:50:35.968557 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.968539 2554 scope.go:117] "RemoveContainer" containerID="69cb885649e51341e17684555d99de6d40046ff7c62ac370e953cd6f3af90229" Sep 08 08:50:35.975709 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.975690 2554 scope.go:117] "RemoveContainer" containerID="775efc4fdffe6b9dc63cd8a95578bec27b20397cc0151cc68ff00322e27016c0" Sep 08 08:50:35.982279 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.982251 2554 scope.go:117] "RemoveContainer" containerID="80d891d01bb50b68732f0b836274125b0b624dcd873662a8fb1677cfa5a22df3" Sep 08 08:50:35.985200 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.985180 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 08 08:50:35.989121 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.989097 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 08 08:50:35.989997 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.989965 2554 scope.go:117] "RemoveContainer" containerID="e9d5dd0af4b16cf638a40321a421154b253df56b983903dd6b85a05eb190da6c" Sep 08 08:50:35.999119 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:35.999102 2554 scope.go:117] "RemoveContainer" containerID="bc29e2817a8aeade8af3b34516144f38407d6e78be296cda978e63f59dbf63d1" Sep 08 08:50:36.006251 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.006236 2554 scope.go:117] "RemoveContainer" containerID="0fe24dc0ceab0d1aca3e3f3d62d46331c3b51bd44b9e4d1090899b50d652d6ea" Sep 08 08:50:36.013015 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.012995 2554 scope.go:117] "RemoveContainer" containerID="61c18b80d9c399ef85945ef84ebf0bbb6c490ee1762bac1fd4eb01ceffd0afc7" Sep 08 08:50:36.013274 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:50:36.013255 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61c18b80d9c399ef85945ef84ebf0bbb6c490ee1762bac1fd4eb01ceffd0afc7\": container with ID starting with 61c18b80d9c399ef85945ef84ebf0bbb6c490ee1762bac1fd4eb01ceffd0afc7 not found: ID does not exist" containerID="61c18b80d9c399ef85945ef84ebf0bbb6c490ee1762bac1fd4eb01ceffd0afc7" Sep 08 08:50:36.013353 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.013287 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61c18b80d9c399ef85945ef84ebf0bbb6c490ee1762bac1fd4eb01ceffd0afc7"} err="failed to get container status \"61c18b80d9c399ef85945ef84ebf0bbb6c490ee1762bac1fd4eb01ceffd0afc7\": rpc error: code = NotFound desc = could not find container \"61c18b80d9c399ef85945ef84ebf0bbb6c490ee1762bac1fd4eb01ceffd0afc7\": container with ID starting with 61c18b80d9c399ef85945ef84ebf0bbb6c490ee1762bac1fd4eb01ceffd0afc7 not found: ID does not exist" Sep 08 08:50:36.013353 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.013311 2554 scope.go:117] "RemoveContainer" containerID="69cb885649e51341e17684555d99de6d40046ff7c62ac370e953cd6f3af90229" Sep 08 08:50:36.013573 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:50:36.013555 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69cb885649e51341e17684555d99de6d40046ff7c62ac370e953cd6f3af90229\": container with ID starting with 69cb885649e51341e17684555d99de6d40046ff7c62ac370e953cd6f3af90229 not found: ID does not exist" containerID="69cb885649e51341e17684555d99de6d40046ff7c62ac370e953cd6f3af90229" Sep 08 08:50:36.013617 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.013580 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69cb885649e51341e17684555d99de6d40046ff7c62ac370e953cd6f3af90229"} err="failed to get container status \"69cb885649e51341e17684555d99de6d40046ff7c62ac370e953cd6f3af90229\": rpc error: code = NotFound desc = could not find container \"69cb885649e51341e17684555d99de6d40046ff7c62ac370e953cd6f3af90229\": container with ID starting with 69cb885649e51341e17684555d99de6d40046ff7c62ac370e953cd6f3af90229 not found: ID does not exist" Sep 08 08:50:36.013617 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.013597 2554 scope.go:117] "RemoveContainer" containerID="775efc4fdffe6b9dc63cd8a95578bec27b20397cc0151cc68ff00322e27016c0" Sep 08 08:50:36.013708 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.013690 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 08 08:50:36.013851 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:50:36.013823 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"775efc4fdffe6b9dc63cd8a95578bec27b20397cc0151cc68ff00322e27016c0\": container with ID starting with 775efc4fdffe6b9dc63cd8a95578bec27b20397cc0151cc68ff00322e27016c0 not found: ID does not exist" containerID="775efc4fdffe6b9dc63cd8a95578bec27b20397cc0151cc68ff00322e27016c0" Sep 08 08:50:36.013946 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.013853 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"775efc4fdffe6b9dc63cd8a95578bec27b20397cc0151cc68ff00322e27016c0"} err="failed to get container status \"775efc4fdffe6b9dc63cd8a95578bec27b20397cc0151cc68ff00322e27016c0\": rpc error: code = NotFound desc = could not find container \"775efc4fdffe6b9dc63cd8a95578bec27b20397cc0151cc68ff00322e27016c0\": container with ID starting with 775efc4fdffe6b9dc63cd8a95578bec27b20397cc0151cc68ff00322e27016c0 not found: ID does not exist" Sep 08 08:50:36.013946 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.013868 2554 scope.go:117] "RemoveContainer" containerID="80d891d01bb50b68732f0b836274125b0b624dcd873662a8fb1677cfa5a22df3" Sep 08 08:50:36.014081 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.014039 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="140a435b-031f-4bc2-96a2-176063bb9b3d" containerName="init-config-reloader" Sep 08 08:50:36.014081 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.014054 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="140a435b-031f-4bc2-96a2-176063bb9b3d" containerName="init-config-reloader" Sep 08 08:50:36.014081 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.014066 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="140a435b-031f-4bc2-96a2-176063bb9b3d" containerName="config-reloader" Sep 08 08:50:36.014081 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.014074 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="140a435b-031f-4bc2-96a2-176063bb9b3d" containerName="config-reloader" Sep 08 08:50:36.014256 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.014094 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="140a435b-031f-4bc2-96a2-176063bb9b3d" containerName="prometheus" Sep 08 08:50:36.014256 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.014102 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="140a435b-031f-4bc2-96a2-176063bb9b3d" containerName="prometheus" Sep 08 08:50:36.014256 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.014112 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="140a435b-031f-4bc2-96a2-176063bb9b3d" containerName="thanos-sidecar" Sep 08 08:50:36.014256 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.014117 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="140a435b-031f-4bc2-96a2-176063bb9b3d" containerName="thanos-sidecar" Sep 08 08:50:36.014256 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.014123 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="140a435b-031f-4bc2-96a2-176063bb9b3d" containerName="kube-rbac-proxy-thanos" Sep 08 08:50:36.014256 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.014128 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="140a435b-031f-4bc2-96a2-176063bb9b3d" containerName="kube-rbac-proxy-thanos" Sep 08 08:50:36.014256 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:50:36.014131 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80d891d01bb50b68732f0b836274125b0b624dcd873662a8fb1677cfa5a22df3\": container with ID starting with 80d891d01bb50b68732f0b836274125b0b624dcd873662a8fb1677cfa5a22df3 not found: ID does not exist" containerID="80d891d01bb50b68732f0b836274125b0b624dcd873662a8fb1677cfa5a22df3" Sep 08 08:50:36.014256 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.014163 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80d891d01bb50b68732f0b836274125b0b624dcd873662a8fb1677cfa5a22df3"} err="failed to get container status \"80d891d01bb50b68732f0b836274125b0b624dcd873662a8fb1677cfa5a22df3\": rpc error: code = NotFound desc = could not find container \"80d891d01bb50b68732f0b836274125b0b624dcd873662a8fb1677cfa5a22df3\": container with ID starting with 80d891d01bb50b68732f0b836274125b0b624dcd873662a8fb1677cfa5a22df3 not found: ID does not exist" Sep 08 08:50:36.014256 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.014176 2554 scope.go:117] "RemoveContainer" containerID="e9d5dd0af4b16cf638a40321a421154b253df56b983903dd6b85a05eb190da6c" Sep 08 08:50:36.014256 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.014138 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="140a435b-031f-4bc2-96a2-176063bb9b3d" containerName="kube-rbac-proxy-web" Sep 08 08:50:36.014256 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.014221 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="140a435b-031f-4bc2-96a2-176063bb9b3d" containerName="kube-rbac-proxy-web" Sep 08 08:50:36.014256 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.014243 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="140a435b-031f-4bc2-96a2-176063bb9b3d" containerName="kube-rbac-proxy" Sep 08 08:50:36.014256 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.014250 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="140a435b-031f-4bc2-96a2-176063bb9b3d" containerName="kube-rbac-proxy" Sep 08 08:50:36.014848 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.014344 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="140a435b-031f-4bc2-96a2-176063bb9b3d" containerName="thanos-sidecar" Sep 08 08:50:36.014848 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.014355 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="140a435b-031f-4bc2-96a2-176063bb9b3d" containerName="kube-rbac-proxy" Sep 08 08:50:36.014848 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.014361 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="140a435b-031f-4bc2-96a2-176063bb9b3d" containerName="kube-rbac-proxy-web" Sep 08 08:50:36.014848 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.014368 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="140a435b-031f-4bc2-96a2-176063bb9b3d" containerName="prometheus" Sep 08 08:50:36.014848 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.014373 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="140a435b-031f-4bc2-96a2-176063bb9b3d" containerName="kube-rbac-proxy-thanos" Sep 08 08:50:36.014848 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.014384 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="140a435b-031f-4bc2-96a2-176063bb9b3d" containerName="config-reloader" Sep 08 08:50:36.014848 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:50:36.014390 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9d5dd0af4b16cf638a40321a421154b253df56b983903dd6b85a05eb190da6c\": container with ID starting with e9d5dd0af4b16cf638a40321a421154b253df56b983903dd6b85a05eb190da6c not found: ID does not exist" containerID="e9d5dd0af4b16cf638a40321a421154b253df56b983903dd6b85a05eb190da6c" Sep 08 08:50:36.014848 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.014415 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9d5dd0af4b16cf638a40321a421154b253df56b983903dd6b85a05eb190da6c"} err="failed to get container status \"e9d5dd0af4b16cf638a40321a421154b253df56b983903dd6b85a05eb190da6c\": rpc error: code = NotFound desc = could not find container \"e9d5dd0af4b16cf638a40321a421154b253df56b983903dd6b85a05eb190da6c\": container with ID starting with e9d5dd0af4b16cf638a40321a421154b253df56b983903dd6b85a05eb190da6c not found: ID does not exist" Sep 08 08:50:36.014848 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.014430 2554 scope.go:117] "RemoveContainer" containerID="bc29e2817a8aeade8af3b34516144f38407d6e78be296cda978e63f59dbf63d1" Sep 08 08:50:36.014848 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:50:36.014629 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc29e2817a8aeade8af3b34516144f38407d6e78be296cda978e63f59dbf63d1\": container with ID starting with bc29e2817a8aeade8af3b34516144f38407d6e78be296cda978e63f59dbf63d1 not found: ID does not exist" containerID="bc29e2817a8aeade8af3b34516144f38407d6e78be296cda978e63f59dbf63d1" Sep 08 08:50:36.014848 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.014646 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc29e2817a8aeade8af3b34516144f38407d6e78be296cda978e63f59dbf63d1"} err="failed to get container status \"bc29e2817a8aeade8af3b34516144f38407d6e78be296cda978e63f59dbf63d1\": rpc error: code = NotFound desc = could not find container \"bc29e2817a8aeade8af3b34516144f38407d6e78be296cda978e63f59dbf63d1\": container with ID starting with bc29e2817a8aeade8af3b34516144f38407d6e78be296cda978e63f59dbf63d1 not found: ID does not exist" Sep 08 08:50:36.014848 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.014659 2554 scope.go:117] "RemoveContainer" containerID="0fe24dc0ceab0d1aca3e3f3d62d46331c3b51bd44b9e4d1090899b50d652d6ea" Sep 08 08:50:36.015412 ip-10-0-141-177 kubenswrapper[2554]: E0908 08:50:36.014873 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fe24dc0ceab0d1aca3e3f3d62d46331c3b51bd44b9e4d1090899b50d652d6ea\": container with ID starting with 0fe24dc0ceab0d1aca3e3f3d62d46331c3b51bd44b9e4d1090899b50d652d6ea not found: ID does not exist" containerID="0fe24dc0ceab0d1aca3e3f3d62d46331c3b51bd44b9e4d1090899b50d652d6ea" Sep 08 08:50:36.015412 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.014896 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fe24dc0ceab0d1aca3e3f3d62d46331c3b51bd44b9e4d1090899b50d652d6ea"} err="failed to get container status \"0fe24dc0ceab0d1aca3e3f3d62d46331c3b51bd44b9e4d1090899b50d652d6ea\": rpc error: code = NotFound desc = could not find container \"0fe24dc0ceab0d1aca3e3f3d62d46331c3b51bd44b9e4d1090899b50d652d6ea\": container with ID starting with 0fe24dc0ceab0d1aca3e3f3d62d46331c3b51bd44b9e4d1090899b50d652d6ea not found: ID does not exist" Sep 08 08:50:36.015412 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.014911 2554 scope.go:117] "RemoveContainer" containerID="61c18b80d9c399ef85945ef84ebf0bbb6c490ee1762bac1fd4eb01ceffd0afc7" Sep 08 08:50:36.015412 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.015184 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61c18b80d9c399ef85945ef84ebf0bbb6c490ee1762bac1fd4eb01ceffd0afc7"} err="failed to get container status \"61c18b80d9c399ef85945ef84ebf0bbb6c490ee1762bac1fd4eb01ceffd0afc7\": rpc error: code = NotFound desc = could not find container \"61c18b80d9c399ef85945ef84ebf0bbb6c490ee1762bac1fd4eb01ceffd0afc7\": container with ID starting with 61c18b80d9c399ef85945ef84ebf0bbb6c490ee1762bac1fd4eb01ceffd0afc7 not found: ID does not exist" Sep 08 08:50:36.015412 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.015201 2554 scope.go:117] "RemoveContainer" containerID="69cb885649e51341e17684555d99de6d40046ff7c62ac370e953cd6f3af90229" Sep 08 08:50:36.015647 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.015436 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69cb885649e51341e17684555d99de6d40046ff7c62ac370e953cd6f3af90229"} err="failed to get container status \"69cb885649e51341e17684555d99de6d40046ff7c62ac370e953cd6f3af90229\": rpc error: code = NotFound desc = could not find container \"69cb885649e51341e17684555d99de6d40046ff7c62ac370e953cd6f3af90229\": container with ID starting with 69cb885649e51341e17684555d99de6d40046ff7c62ac370e953cd6f3af90229 not found: ID does not exist" Sep 08 08:50:36.015647 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.015453 2554 scope.go:117] "RemoveContainer" containerID="775efc4fdffe6b9dc63cd8a95578bec27b20397cc0151cc68ff00322e27016c0" Sep 08 08:50:36.015746 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.015656 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"775efc4fdffe6b9dc63cd8a95578bec27b20397cc0151cc68ff00322e27016c0"} err="failed to get container status \"775efc4fdffe6b9dc63cd8a95578bec27b20397cc0151cc68ff00322e27016c0\": rpc error: code = NotFound desc = could not find container \"775efc4fdffe6b9dc63cd8a95578bec27b20397cc0151cc68ff00322e27016c0\": container with ID starting with 775efc4fdffe6b9dc63cd8a95578bec27b20397cc0151cc68ff00322e27016c0 not found: ID does not exist" Sep 08 08:50:36.015746 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.015676 2554 scope.go:117] "RemoveContainer" containerID="80d891d01bb50b68732f0b836274125b0b624dcd873662a8fb1677cfa5a22df3" Sep 08 08:50:36.015940 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.015917 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80d891d01bb50b68732f0b836274125b0b624dcd873662a8fb1677cfa5a22df3"} err="failed to get container status \"80d891d01bb50b68732f0b836274125b0b624dcd873662a8fb1677cfa5a22df3\": rpc error: code = NotFound desc = could not find container \"80d891d01bb50b68732f0b836274125b0b624dcd873662a8fb1677cfa5a22df3\": container with ID starting with 80d891d01bb50b68732f0b836274125b0b624dcd873662a8fb1677cfa5a22df3 not found: ID does not exist" Sep 08 08:50:36.016009 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.015941 2554 scope.go:117] "RemoveContainer" containerID="e9d5dd0af4b16cf638a40321a421154b253df56b983903dd6b85a05eb190da6c" Sep 08 08:50:36.016184 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.016164 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9d5dd0af4b16cf638a40321a421154b253df56b983903dd6b85a05eb190da6c"} err="failed to get container status \"e9d5dd0af4b16cf638a40321a421154b253df56b983903dd6b85a05eb190da6c\": rpc error: code = NotFound desc = could not find container \"e9d5dd0af4b16cf638a40321a421154b253df56b983903dd6b85a05eb190da6c\": container with ID starting with e9d5dd0af4b16cf638a40321a421154b253df56b983903dd6b85a05eb190da6c not found: ID does not exist" Sep 08 08:50:36.016184 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.016183 2554 scope.go:117] "RemoveContainer" containerID="bc29e2817a8aeade8af3b34516144f38407d6e78be296cda978e63f59dbf63d1" Sep 08 08:50:36.016524 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.016503 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc29e2817a8aeade8af3b34516144f38407d6e78be296cda978e63f59dbf63d1"} err="failed to get container status \"bc29e2817a8aeade8af3b34516144f38407d6e78be296cda978e63f59dbf63d1\": rpc error: code = NotFound desc = could not find container \"bc29e2817a8aeade8af3b34516144f38407d6e78be296cda978e63f59dbf63d1\": container with ID starting with bc29e2817a8aeade8af3b34516144f38407d6e78be296cda978e63f59dbf63d1 not found: ID does not exist" Sep 08 08:50:36.016524 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.016524 2554 scope.go:117] "RemoveContainer" containerID="0fe24dc0ceab0d1aca3e3f3d62d46331c3b51bd44b9e4d1090899b50d652d6ea" Sep 08 08:50:36.017300 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.017260 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fe24dc0ceab0d1aca3e3f3d62d46331c3b51bd44b9e4d1090899b50d652d6ea"} err="failed to get container status \"0fe24dc0ceab0d1aca3e3f3d62d46331c3b51bd44b9e4d1090899b50d652d6ea\": rpc error: code = NotFound desc = could not find container \"0fe24dc0ceab0d1aca3e3f3d62d46331c3b51bd44b9e4d1090899b50d652d6ea\": container with ID starting with 0fe24dc0ceab0d1aca3e3f3d62d46331c3b51bd44b9e4d1090899b50d652d6ea not found: ID does not exist" Sep 08 08:50:36.017300 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.017298 2554 scope.go:117] "RemoveContainer" containerID="61c18b80d9c399ef85945ef84ebf0bbb6c490ee1762bac1fd4eb01ceffd0afc7" Sep 08 08:50:36.017601 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.017575 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61c18b80d9c399ef85945ef84ebf0bbb6c490ee1762bac1fd4eb01ceffd0afc7"} err="failed to get container status \"61c18b80d9c399ef85945ef84ebf0bbb6c490ee1762bac1fd4eb01ceffd0afc7\": rpc error: code = NotFound desc = could not find container \"61c18b80d9c399ef85945ef84ebf0bbb6c490ee1762bac1fd4eb01ceffd0afc7\": container with ID starting with 61c18b80d9c399ef85945ef84ebf0bbb6c490ee1762bac1fd4eb01ceffd0afc7 not found: ID does not exist" Sep 08 08:50:36.017601 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.017601 2554 scope.go:117] "RemoveContainer" containerID="69cb885649e51341e17684555d99de6d40046ff7c62ac370e953cd6f3af90229" Sep 08 08:50:36.017925 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.017891 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69cb885649e51341e17684555d99de6d40046ff7c62ac370e953cd6f3af90229"} err="failed to get container status \"69cb885649e51341e17684555d99de6d40046ff7c62ac370e953cd6f3af90229\": rpc error: code = NotFound desc = could not find container \"69cb885649e51341e17684555d99de6d40046ff7c62ac370e953cd6f3af90229\": container with ID starting with 69cb885649e51341e17684555d99de6d40046ff7c62ac370e953cd6f3af90229 not found: ID does not exist" Sep 08 08:50:36.017925 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.017926 2554 scope.go:117] "RemoveContainer" containerID="775efc4fdffe6b9dc63cd8a95578bec27b20397cc0151cc68ff00322e27016c0" Sep 08 08:50:36.018213 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.018189 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.018352 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.018328 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"775efc4fdffe6b9dc63cd8a95578bec27b20397cc0151cc68ff00322e27016c0"} err="failed to get container status \"775efc4fdffe6b9dc63cd8a95578bec27b20397cc0151cc68ff00322e27016c0\": rpc error: code = NotFound desc = could not find container \"775efc4fdffe6b9dc63cd8a95578bec27b20397cc0151cc68ff00322e27016c0\": container with ID starting with 775efc4fdffe6b9dc63cd8a95578bec27b20397cc0151cc68ff00322e27016c0 not found: ID does not exist" Sep 08 08:50:36.018352 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.018355 2554 scope.go:117] "RemoveContainer" containerID="80d891d01bb50b68732f0b836274125b0b624dcd873662a8fb1677cfa5a22df3" Sep 08 08:50:36.019397 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.019367 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80d891d01bb50b68732f0b836274125b0b624dcd873662a8fb1677cfa5a22df3"} err="failed to get container status \"80d891d01bb50b68732f0b836274125b0b624dcd873662a8fb1677cfa5a22df3\": rpc error: code = NotFound desc = could not find container \"80d891d01bb50b68732f0b836274125b0b624dcd873662a8fb1677cfa5a22df3\": container with ID starting with 80d891d01bb50b68732f0b836274125b0b624dcd873662a8fb1677cfa5a22df3 not found: ID does not exist" Sep 08 08:50:36.019502 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.019396 2554 scope.go:117] "RemoveContainer" containerID="e9d5dd0af4b16cf638a40321a421154b253df56b983903dd6b85a05eb190da6c" Sep 08 08:50:36.019812 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.019765 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9d5dd0af4b16cf638a40321a421154b253df56b983903dd6b85a05eb190da6c"} err="failed to get container status \"e9d5dd0af4b16cf638a40321a421154b253df56b983903dd6b85a05eb190da6c\": rpc error: code = NotFound desc = could not find container \"e9d5dd0af4b16cf638a40321a421154b253df56b983903dd6b85a05eb190da6c\": container with ID starting with e9d5dd0af4b16cf638a40321a421154b253df56b983903dd6b85a05eb190da6c not found: ID does not exist" Sep 08 08:50:36.019908 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.019813 2554 scope.go:117] "RemoveContainer" containerID="bc29e2817a8aeade8af3b34516144f38407d6e78be296cda978e63f59dbf63d1" Sep 08 08:50:36.020284 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.020253 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc29e2817a8aeade8af3b34516144f38407d6e78be296cda978e63f59dbf63d1"} err="failed to get container status \"bc29e2817a8aeade8af3b34516144f38407d6e78be296cda978e63f59dbf63d1\": rpc error: code = NotFound desc = could not find container \"bc29e2817a8aeade8af3b34516144f38407d6e78be296cda978e63f59dbf63d1\": container with ID starting with bc29e2817a8aeade8af3b34516144f38407d6e78be296cda978e63f59dbf63d1 not found: ID does not exist" Sep 08 08:50:36.020370 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.020285 2554 scope.go:117] "RemoveContainer" containerID="0fe24dc0ceab0d1aca3e3f3d62d46331c3b51bd44b9e4d1090899b50d652d6ea" Sep 08 08:50:36.020705 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.020687 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 08 08:50:36.020798 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.020750 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fe24dc0ceab0d1aca3e3f3d62d46331c3b51bd44b9e4d1090899b50d652d6ea"} err="failed to get container status \"0fe24dc0ceab0d1aca3e3f3d62d46331c3b51bd44b9e4d1090899b50d652d6ea\": rpc error: code = NotFound desc = could not find container \"0fe24dc0ceab0d1aca3e3f3d62d46331c3b51bd44b9e4d1090899b50d652d6ea\": container with ID starting with 0fe24dc0ceab0d1aca3e3f3d62d46331c3b51bd44b9e4d1090899b50d652d6ea not found: ID does not exist" Sep 08 08:50:36.020798 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.020770 2554 scope.go:117] "RemoveContainer" containerID="61c18b80d9c399ef85945ef84ebf0bbb6c490ee1762bac1fd4eb01ceffd0afc7" Sep 08 08:50:36.021047 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.021030 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 08 08:50:36.021188 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.021037 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 08 08:50:36.021286 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.021162 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61c18b80d9c399ef85945ef84ebf0bbb6c490ee1762bac1fd4eb01ceffd0afc7"} err="failed to get container status \"61c18b80d9c399ef85945ef84ebf0bbb6c490ee1762bac1fd4eb01ceffd0afc7\": rpc error: code = NotFound desc = could not find container \"61c18b80d9c399ef85945ef84ebf0bbb6c490ee1762bac1fd4eb01ceffd0afc7\": container with ID starting with 61c18b80d9c399ef85945ef84ebf0bbb6c490ee1762bac1fd4eb01ceffd0afc7 not found: ID does not exist" Sep 08 08:50:36.021286 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.021211 2554 scope.go:117] "RemoveContainer" containerID="69cb885649e51341e17684555d99de6d40046ff7c62ac370e953cd6f3af90229" Sep 08 08:50:36.021286 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.021241 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 08 08:50:36.021518 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.021489 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 08 08:50:36.021518 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.021507 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 08 08:50:36.021657 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.021532 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 08 08:50:36.021657 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.021566 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-blm64lqjltb67\"" Sep 08 08:50:36.021657 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.021489 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 08 08:50:36.021804 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.021668 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69cb885649e51341e17684555d99de6d40046ff7c62ac370e953cd6f3af90229"} err="failed to get container status \"69cb885649e51341e17684555d99de6d40046ff7c62ac370e953cd6f3af90229\": rpc error: code = NotFound desc = could not find container \"69cb885649e51341e17684555d99de6d40046ff7c62ac370e953cd6f3af90229\": container with ID starting with 69cb885649e51341e17684555d99de6d40046ff7c62ac370e953cd6f3af90229 not found: ID does not exist" Sep 08 08:50:36.021804 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.021689 2554 scope.go:117] "RemoveContainer" containerID="775efc4fdffe6b9dc63cd8a95578bec27b20397cc0151cc68ff00322e27016c0" Sep 08 08:50:36.021909 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.021832 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 08 08:50:36.021909 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.021860 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 08 08:50:36.022133 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.021972 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 08 08:50:36.022133 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.022020 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"775efc4fdffe6b9dc63cd8a95578bec27b20397cc0151cc68ff00322e27016c0"} err="failed to get container status \"775efc4fdffe6b9dc63cd8a95578bec27b20397cc0151cc68ff00322e27016c0\": rpc error: code = NotFound desc = could not find container \"775efc4fdffe6b9dc63cd8a95578bec27b20397cc0151cc68ff00322e27016c0\": container with ID starting with 775efc4fdffe6b9dc63cd8a95578bec27b20397cc0151cc68ff00322e27016c0 not found: ID does not exist" Sep 08 08:50:36.022133 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.022054 2554 scope.go:117] "RemoveContainer" containerID="80d891d01bb50b68732f0b836274125b0b624dcd873662a8fb1677cfa5a22df3" Sep 08 08:50:36.022375 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.022149 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 08 08:50:36.022375 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.022164 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 08 08:50:36.022375 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.022328 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-gmmlb\"" Sep 08 08:50:36.022586 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.022466 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80d891d01bb50b68732f0b836274125b0b624dcd873662a8fb1677cfa5a22df3"} err="failed to get container status \"80d891d01bb50b68732f0b836274125b0b624dcd873662a8fb1677cfa5a22df3\": rpc error: code = NotFound desc = could not find container \"80d891d01bb50b68732f0b836274125b0b624dcd873662a8fb1677cfa5a22df3\": container with ID starting with 80d891d01bb50b68732f0b836274125b0b624dcd873662a8fb1677cfa5a22df3 not found: ID does not exist" Sep 08 08:50:36.022586 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.022487 2554 scope.go:117] "RemoveContainer" containerID="e9d5dd0af4b16cf638a40321a421154b253df56b983903dd6b85a05eb190da6c" Sep 08 08:50:36.022816 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.022745 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9d5dd0af4b16cf638a40321a421154b253df56b983903dd6b85a05eb190da6c"} err="failed to get container status \"e9d5dd0af4b16cf638a40321a421154b253df56b983903dd6b85a05eb190da6c\": rpc error: code = NotFound desc = could not find container \"e9d5dd0af4b16cf638a40321a421154b253df56b983903dd6b85a05eb190da6c\": container with ID starting with e9d5dd0af4b16cf638a40321a421154b253df56b983903dd6b85a05eb190da6c not found: ID does not exist" Sep 08 08:50:36.022816 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.022765 2554 scope.go:117] "RemoveContainer" containerID="bc29e2817a8aeade8af3b34516144f38407d6e78be296cda978e63f59dbf63d1" Sep 08 08:50:36.023126 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.023061 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc29e2817a8aeade8af3b34516144f38407d6e78be296cda978e63f59dbf63d1"} err="failed to get container status \"bc29e2817a8aeade8af3b34516144f38407d6e78be296cda978e63f59dbf63d1\": rpc error: code = NotFound desc = could not find container \"bc29e2817a8aeade8af3b34516144f38407d6e78be296cda978e63f59dbf63d1\": container with ID starting with bc29e2817a8aeade8af3b34516144f38407d6e78be296cda978e63f59dbf63d1 not found: ID does not exist" Sep 08 08:50:36.023126 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.023078 2554 scope.go:117] "RemoveContainer" containerID="0fe24dc0ceab0d1aca3e3f3d62d46331c3b51bd44b9e4d1090899b50d652d6ea" Sep 08 08:50:36.023445 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.023337 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fe24dc0ceab0d1aca3e3f3d62d46331c3b51bd44b9e4d1090899b50d652d6ea"} err="failed to get container status \"0fe24dc0ceab0d1aca3e3f3d62d46331c3b51bd44b9e4d1090899b50d652d6ea\": rpc error: code = NotFound desc = could not find container \"0fe24dc0ceab0d1aca3e3f3d62d46331c3b51bd44b9e4d1090899b50d652d6ea\": container with ID starting with 0fe24dc0ceab0d1aca3e3f3d62d46331c3b51bd44b9e4d1090899b50d652d6ea not found: ID does not exist" Sep 08 08:50:36.023445 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.023360 2554 scope.go:117] "RemoveContainer" containerID="61c18b80d9c399ef85945ef84ebf0bbb6c490ee1762bac1fd4eb01ceffd0afc7" Sep 08 08:50:36.023725 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.023620 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61c18b80d9c399ef85945ef84ebf0bbb6c490ee1762bac1fd4eb01ceffd0afc7"} err="failed to get container status \"61c18b80d9c399ef85945ef84ebf0bbb6c490ee1762bac1fd4eb01ceffd0afc7\": rpc error: code = NotFound desc = could not find container \"61c18b80d9c399ef85945ef84ebf0bbb6c490ee1762bac1fd4eb01ceffd0afc7\": container with ID starting with 61c18b80d9c399ef85945ef84ebf0bbb6c490ee1762bac1fd4eb01ceffd0afc7 not found: ID does not exist" Sep 08 08:50:36.023725 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.023638 2554 scope.go:117] "RemoveContainer" containerID="69cb885649e51341e17684555d99de6d40046ff7c62ac370e953cd6f3af90229" Sep 08 08:50:36.023896 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.023822 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 08 08:50:36.023896 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.023872 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69cb885649e51341e17684555d99de6d40046ff7c62ac370e953cd6f3af90229"} err="failed to get container status \"69cb885649e51341e17684555d99de6d40046ff7c62ac370e953cd6f3af90229\": rpc error: code = NotFound desc = could not find container \"69cb885649e51341e17684555d99de6d40046ff7c62ac370e953cd6f3af90229\": container with ID starting with 69cb885649e51341e17684555d99de6d40046ff7c62ac370e953cd6f3af90229 not found: ID does not exist" Sep 08 08:50:36.024036 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.023894 2554 scope.go:117] "RemoveContainer" containerID="775efc4fdffe6b9dc63cd8a95578bec27b20397cc0151cc68ff00322e27016c0" Sep 08 08:50:36.024218 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.024194 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"775efc4fdffe6b9dc63cd8a95578bec27b20397cc0151cc68ff00322e27016c0"} err="failed to get container status \"775efc4fdffe6b9dc63cd8a95578bec27b20397cc0151cc68ff00322e27016c0\": rpc error: code = NotFound desc = could not find container \"775efc4fdffe6b9dc63cd8a95578bec27b20397cc0151cc68ff00322e27016c0\": container with ID starting with 775efc4fdffe6b9dc63cd8a95578bec27b20397cc0151cc68ff00322e27016c0 not found: ID does not exist" Sep 08 08:50:36.024292 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.024221 2554 scope.go:117] "RemoveContainer" containerID="80d891d01bb50b68732f0b836274125b0b624dcd873662a8fb1677cfa5a22df3" Sep 08 08:50:36.024532 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.024499 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80d891d01bb50b68732f0b836274125b0b624dcd873662a8fb1677cfa5a22df3"} err="failed to get container status \"80d891d01bb50b68732f0b836274125b0b624dcd873662a8fb1677cfa5a22df3\": rpc error: code = NotFound desc = could not find container \"80d891d01bb50b68732f0b836274125b0b624dcd873662a8fb1677cfa5a22df3\": container with ID starting with 80d891d01bb50b68732f0b836274125b0b624dcd873662a8fb1677cfa5a22df3 not found: ID does not exist" Sep 08 08:50:36.024532 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.024520 2554 scope.go:117] "RemoveContainer" containerID="e9d5dd0af4b16cf638a40321a421154b253df56b983903dd6b85a05eb190da6c" Sep 08 08:50:36.024815 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.024789 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9d5dd0af4b16cf638a40321a421154b253df56b983903dd6b85a05eb190da6c"} err="failed to get container status \"e9d5dd0af4b16cf638a40321a421154b253df56b983903dd6b85a05eb190da6c\": rpc error: code = NotFound desc = could not find container \"e9d5dd0af4b16cf638a40321a421154b253df56b983903dd6b85a05eb190da6c\": container with ID starting with e9d5dd0af4b16cf638a40321a421154b253df56b983903dd6b85a05eb190da6c not found: ID does not exist" Sep 08 08:50:36.024957 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.024816 2554 scope.go:117] "RemoveContainer" containerID="bc29e2817a8aeade8af3b34516144f38407d6e78be296cda978e63f59dbf63d1" Sep 08 08:50:36.025137 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.025103 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc29e2817a8aeade8af3b34516144f38407d6e78be296cda978e63f59dbf63d1"} err="failed to get container status \"bc29e2817a8aeade8af3b34516144f38407d6e78be296cda978e63f59dbf63d1\": rpc error: code = NotFound desc = could not find container \"bc29e2817a8aeade8af3b34516144f38407d6e78be296cda978e63f59dbf63d1\": container with ID starting with bc29e2817a8aeade8af3b34516144f38407d6e78be296cda978e63f59dbf63d1 not found: ID does not exist" Sep 08 08:50:36.025137 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.025131 2554 scope.go:117] "RemoveContainer" containerID="0fe24dc0ceab0d1aca3e3f3d62d46331c3b51bd44b9e4d1090899b50d652d6ea" Sep 08 08:50:36.025636 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.025458 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fe24dc0ceab0d1aca3e3f3d62d46331c3b51bd44b9e4d1090899b50d652d6ea"} err="failed to get container status \"0fe24dc0ceab0d1aca3e3f3d62d46331c3b51bd44b9e4d1090899b50d652d6ea\": rpc error: code = NotFound desc = could not find container \"0fe24dc0ceab0d1aca3e3f3d62d46331c3b51bd44b9e4d1090899b50d652d6ea\": container with ID starting with 0fe24dc0ceab0d1aca3e3f3d62d46331c3b51bd44b9e4d1090899b50d652d6ea not found: ID does not exist" Sep 08 08:50:36.025636 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.025481 2554 scope.go:117] "RemoveContainer" containerID="61c18b80d9c399ef85945ef84ebf0bbb6c490ee1762bac1fd4eb01ceffd0afc7" Sep 08 08:50:36.025758 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.025712 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61c18b80d9c399ef85945ef84ebf0bbb6c490ee1762bac1fd4eb01ceffd0afc7"} err="failed to get container status \"61c18b80d9c399ef85945ef84ebf0bbb6c490ee1762bac1fd4eb01ceffd0afc7\": rpc error: code = NotFound desc = could not find container \"61c18b80d9c399ef85945ef84ebf0bbb6c490ee1762bac1fd4eb01ceffd0afc7\": container with ID starting with 61c18b80d9c399ef85945ef84ebf0bbb6c490ee1762bac1fd4eb01ceffd0afc7 not found: ID does not exist" Sep 08 08:50:36.025758 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.025736 2554 scope.go:117] "RemoveContainer" containerID="69cb885649e51341e17684555d99de6d40046ff7c62ac370e953cd6f3af90229" Sep 08 08:50:36.026272 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.026244 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69cb885649e51341e17684555d99de6d40046ff7c62ac370e953cd6f3af90229"} err="failed to get container status \"69cb885649e51341e17684555d99de6d40046ff7c62ac370e953cd6f3af90229\": rpc error: code = NotFound desc = could not find container \"69cb885649e51341e17684555d99de6d40046ff7c62ac370e953cd6f3af90229\": container with ID starting with 69cb885649e51341e17684555d99de6d40046ff7c62ac370e953cd6f3af90229 not found: ID does not exist" Sep 08 08:50:36.026272 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.026264 2554 scope.go:117] "RemoveContainer" containerID="775efc4fdffe6b9dc63cd8a95578bec27b20397cc0151cc68ff00322e27016c0" Sep 08 08:50:36.026628 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.026568 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"775efc4fdffe6b9dc63cd8a95578bec27b20397cc0151cc68ff00322e27016c0"} err="failed to get container status \"775efc4fdffe6b9dc63cd8a95578bec27b20397cc0151cc68ff00322e27016c0\": rpc error: code = NotFound desc = could not find container \"775efc4fdffe6b9dc63cd8a95578bec27b20397cc0151cc68ff00322e27016c0\": container with ID starting with 775efc4fdffe6b9dc63cd8a95578bec27b20397cc0151cc68ff00322e27016c0 not found: ID does not exist" Sep 08 08:50:36.026628 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.026601 2554 scope.go:117] "RemoveContainer" containerID="80d891d01bb50b68732f0b836274125b0b624dcd873662a8fb1677cfa5a22df3" Sep 08 08:50:36.027002 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.026882 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80d891d01bb50b68732f0b836274125b0b624dcd873662a8fb1677cfa5a22df3"} err="failed to get container status \"80d891d01bb50b68732f0b836274125b0b624dcd873662a8fb1677cfa5a22df3\": rpc error: code = NotFound desc = could not find container \"80d891d01bb50b68732f0b836274125b0b624dcd873662a8fb1677cfa5a22df3\": container with ID starting with 80d891d01bb50b68732f0b836274125b0b624dcd873662a8fb1677cfa5a22df3 not found: ID does not exist" Sep 08 08:50:36.027002 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.026907 2554 scope.go:117] "RemoveContainer" containerID="e9d5dd0af4b16cf638a40321a421154b253df56b983903dd6b85a05eb190da6c" Sep 08 08:50:36.027275 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.027249 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9d5dd0af4b16cf638a40321a421154b253df56b983903dd6b85a05eb190da6c"} err="failed to get container status \"e9d5dd0af4b16cf638a40321a421154b253df56b983903dd6b85a05eb190da6c\": rpc error: code = NotFound desc = could not find container \"e9d5dd0af4b16cf638a40321a421154b253df56b983903dd6b85a05eb190da6c\": container with ID starting with e9d5dd0af4b16cf638a40321a421154b253df56b983903dd6b85a05eb190da6c not found: ID does not exist" Sep 08 08:50:36.027356 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.027277 2554 scope.go:117] "RemoveContainer" containerID="bc29e2817a8aeade8af3b34516144f38407d6e78be296cda978e63f59dbf63d1" Sep 08 08:50:36.027758 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.027636 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc29e2817a8aeade8af3b34516144f38407d6e78be296cda978e63f59dbf63d1"} err="failed to get container status \"bc29e2817a8aeade8af3b34516144f38407d6e78be296cda978e63f59dbf63d1\": rpc error: code = NotFound desc = could not find container \"bc29e2817a8aeade8af3b34516144f38407d6e78be296cda978e63f59dbf63d1\": container with ID starting with bc29e2817a8aeade8af3b34516144f38407d6e78be296cda978e63f59dbf63d1 not found: ID does not exist" Sep 08 08:50:36.027758 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.027668 2554 scope.go:117] "RemoveContainer" containerID="0fe24dc0ceab0d1aca3e3f3d62d46331c3b51bd44b9e4d1090899b50d652d6ea" Sep 08 08:50:36.027900 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.027803 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 08 08:50:36.028010 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.027961 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fe24dc0ceab0d1aca3e3f3d62d46331c3b51bd44b9e4d1090899b50d652d6ea"} err="failed to get container status \"0fe24dc0ceab0d1aca3e3f3d62d46331c3b51bd44b9e4d1090899b50d652d6ea\": rpc error: code = NotFound desc = could not find container \"0fe24dc0ceab0d1aca3e3f3d62d46331c3b51bd44b9e4d1090899b50d652d6ea\": container with ID starting with 0fe24dc0ceab0d1aca3e3f3d62d46331c3b51bd44b9e4d1090899b50d652d6ea not found: ID does not exist" Sep 08 08:50:36.033065 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.033043 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="140a435b-031f-4bc2-96a2-176063bb9b3d" path="/var/lib/kubelet/pods/140a435b-031f-4bc2-96a2-176063bb9b3d/volumes" Sep 08 08:50:36.034321 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.034301 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 08 08:50:36.117391 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.117362 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c07b7006-ab2f-4f85-bc2b-3427edcc0923-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.117391 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.117393 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c07b7006-ab2f-4f85-bc2b-3427edcc0923-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.117556 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.117416 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c07b7006-ab2f-4f85-bc2b-3427edcc0923-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.117556 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.117445 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c07b7006-ab2f-4f85-bc2b-3427edcc0923-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.117556 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.117512 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c07b7006-ab2f-4f85-bc2b-3427edcc0923-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.117556 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.117527 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c07b7006-ab2f-4f85-bc2b-3427edcc0923-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.117736 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.117569 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c07b7006-ab2f-4f85-bc2b-3427edcc0923-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.117736 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.117595 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c07b7006-ab2f-4f85-bc2b-3427edcc0923-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.117736 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.117628 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c07b7006-ab2f-4f85-bc2b-3427edcc0923-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.117736 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.117654 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c07b7006-ab2f-4f85-bc2b-3427edcc0923-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.117736 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.117719 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c07b7006-ab2f-4f85-bc2b-3427edcc0923-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.117969 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.117743 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c07b7006-ab2f-4f85-bc2b-3427edcc0923-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.117969 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.117774 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c07b7006-ab2f-4f85-bc2b-3427edcc0923-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.117969 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.117847 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c07b7006-ab2f-4f85-bc2b-3427edcc0923-config-out\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.117969 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.117890 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c07b7006-ab2f-4f85-bc2b-3427edcc0923-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.117969 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.117918 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c07b7006-ab2f-4f85-bc2b-3427edcc0923-web-config\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.117969 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.117944 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtqz8\" (UniqueName: \"kubernetes.io/projected/c07b7006-ab2f-4f85-bc2b-3427edcc0923-kube-api-access-wtqz8\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.118217 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.118012 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c07b7006-ab2f-4f85-bc2b-3427edcc0923-config\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.118217 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.118037 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c07b7006-ab2f-4f85-bc2b-3427edcc0923-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.118217 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.118059 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c07b7006-ab2f-4f85-bc2b-3427edcc0923-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.218469 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.218387 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c07b7006-ab2f-4f85-bc2b-3427edcc0923-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.218469 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.218426 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c07b7006-ab2f-4f85-bc2b-3427edcc0923-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.218469 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.218455 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c07b7006-ab2f-4f85-bc2b-3427edcc0923-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.218689 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.218480 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c07b7006-ab2f-4f85-bc2b-3427edcc0923-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.218689 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.218518 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c07b7006-ab2f-4f85-bc2b-3427edcc0923-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.218689 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.218542 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c07b7006-ab2f-4f85-bc2b-3427edcc0923-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.218689 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.218660 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c07b7006-ab2f-4f85-bc2b-3427edcc0923-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.218877 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.218697 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c07b7006-ab2f-4f85-bc2b-3427edcc0923-config-out\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.218877 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.218731 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c07b7006-ab2f-4f85-bc2b-3427edcc0923-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.218877 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.218757 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c07b7006-ab2f-4f85-bc2b-3427edcc0923-web-config\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.218877 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.218780 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtqz8\" (UniqueName: \"kubernetes.io/projected/c07b7006-ab2f-4f85-bc2b-3427edcc0923-kube-api-access-wtqz8\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.218877 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.218814 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c07b7006-ab2f-4f85-bc2b-3427edcc0923-config\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.218877 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.218837 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c07b7006-ab2f-4f85-bc2b-3427edcc0923-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.218877 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.218860 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c07b7006-ab2f-4f85-bc2b-3427edcc0923-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.219252 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.218896 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c07b7006-ab2f-4f85-bc2b-3427edcc0923-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.219252 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.218923 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c07b7006-ab2f-4f85-bc2b-3427edcc0923-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.219252 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.218945 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c07b7006-ab2f-4f85-bc2b-3427edcc0923-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.219252 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.218973 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c07b7006-ab2f-4f85-bc2b-3427edcc0923-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.219252 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.219050 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c07b7006-ab2f-4f85-bc2b-3427edcc0923-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.219252 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.219073 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c07b7006-ab2f-4f85-bc2b-3427edcc0923-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.219252 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.219170 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c07b7006-ab2f-4f85-bc2b-3427edcc0923-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.219587 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.219291 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c07b7006-ab2f-4f85-bc2b-3427edcc0923-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.219587 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.219414 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c07b7006-ab2f-4f85-bc2b-3427edcc0923-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.219764 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.219705 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c07b7006-ab2f-4f85-bc2b-3427edcc0923-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.220278 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.220243 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c07b7006-ab2f-4f85-bc2b-3427edcc0923-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.220384 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.220250 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c07b7006-ab2f-4f85-bc2b-3427edcc0923-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.221694 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.221666 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c07b7006-ab2f-4f85-bc2b-3427edcc0923-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.222236 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.222164 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c07b7006-ab2f-4f85-bc2b-3427edcc0923-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.222702 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.222616 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/c07b7006-ab2f-4f85-bc2b-3427edcc0923-config\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.222702 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.222684 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c07b7006-ab2f-4f85-bc2b-3427edcc0923-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.222702 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.222694 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c07b7006-ab2f-4f85-bc2b-3427edcc0923-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.222903 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.222716 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c07b7006-ab2f-4f85-bc2b-3427edcc0923-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.223296 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.223276 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c07b7006-ab2f-4f85-bc2b-3427edcc0923-web-config\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.223868 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.223846 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c07b7006-ab2f-4f85-bc2b-3427edcc0923-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.224344 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.224323 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c07b7006-ab2f-4f85-bc2b-3427edcc0923-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.224344 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.224336 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c07b7006-ab2f-4f85-bc2b-3427edcc0923-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.228573 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.225032 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c07b7006-ab2f-4f85-bc2b-3427edcc0923-config-out\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.228573 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.225169 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c07b7006-ab2f-4f85-bc2b-3427edcc0923-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.228573 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.225358 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c07b7006-ab2f-4f85-bc2b-3427edcc0923-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.228573 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.228187 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtqz8\" (UniqueName: \"kubernetes.io/projected/c07b7006-ab2f-4f85-bc2b-3427edcc0923-kube-api-access-wtqz8\") pod \"prometheus-k8s-0\" (UID: \"c07b7006-ab2f-4f85-bc2b-3427edcc0923\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.346206 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.346164 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:50:36.629756 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.629728 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 08 08:50:36.632030 ip-10-0-141-177 kubenswrapper[2554]: W0908 08:50:36.632003 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc07b7006_ab2f_4f85_bc2b_3427edcc0923.slice/crio-dcca4f08e2bd483ae32d49e6ed71eca572163dca3f3a321b0fce80d4f167199e WatchSource:0}: Error finding container dcca4f08e2bd483ae32d49e6ed71eca572163dca3f3a321b0fce80d4f167199e: Status 404 returned error can't find the container with id dcca4f08e2bd483ae32d49e6ed71eca572163dca3f3a321b0fce80d4f167199e Sep 08 08:50:36.968465 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.968427 2554 generic.go:356] "Generic (PLEG): container finished" podID="c07b7006-ab2f-4f85-bc2b-3427edcc0923" containerID="f7e1845f8dc685566a4d918687de22d968518d5c7207e5c02c69a800f462b536" exitCode=0 Sep 08 08:50:36.968946 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.968530 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c07b7006-ab2f-4f85-bc2b-3427edcc0923","Type":"ContainerDied","Data":"f7e1845f8dc685566a4d918687de22d968518d5c7207e5c02c69a800f462b536"} Sep 08 08:50:36.968946 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:36.968628 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c07b7006-ab2f-4f85-bc2b-3427edcc0923","Type":"ContainerStarted","Data":"dcca4f08e2bd483ae32d49e6ed71eca572163dca3f3a321b0fce80d4f167199e"} Sep 08 08:50:37.976321 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:37.976231 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-76497f5b48-g9v24" event={"ID":"9afa9c3f-a12a-4e54-867d-0ff38b8eafa8","Type":"ContainerStarted","Data":"13243938d9afe291c179c3bd1ecd17082eae700834226f840808a7646cf1cc43"} Sep 08 08:50:37.976321 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:37.976301 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-76497f5b48-g9v24" event={"ID":"9afa9c3f-a12a-4e54-867d-0ff38b8eafa8","Type":"ContainerStarted","Data":"4ee661d104e4dd82b221e09214fb75c52afe533dc48207bedb8d36fa0935e66c"} Sep 08 08:50:37.976755 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:37.976330 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-76497f5b48-g9v24" event={"ID":"9afa9c3f-a12a-4e54-867d-0ff38b8eafa8","Type":"ContainerStarted","Data":"e579e477360c4392fcf319f31495a50bc09d1e7afa4424be2c03730ada2c5d01"} Sep 08 08:50:37.979644 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:37.979618 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c07b7006-ab2f-4f85-bc2b-3427edcc0923","Type":"ContainerStarted","Data":"6f176564c9163f7848feee0bf692c7982ebb1a766f67a847997f7599d8859ada"} Sep 08 08:50:37.979742 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:37.979676 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c07b7006-ab2f-4f85-bc2b-3427edcc0923","Type":"ContainerStarted","Data":"901087c4dff436559c8e84bbb2878e88b893f8915486f759b07bc5f52a4847e2"} Sep 08 08:50:37.979742 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:37.979702 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c07b7006-ab2f-4f85-bc2b-3427edcc0923","Type":"ContainerStarted","Data":"24406ba12b8bc75ca689f742306983041bee7281061ad2f962a0ed5d58ddeaf5"} Sep 08 08:50:37.979742 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:37.979726 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c07b7006-ab2f-4f85-bc2b-3427edcc0923","Type":"ContainerStarted","Data":"96032bc83ab38958df1f528f458b39c7d41eaccc256c7672fb7ac02829b7c779"} Sep 08 08:50:37.979845 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:37.979752 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c07b7006-ab2f-4f85-bc2b-3427edcc0923","Type":"ContainerStarted","Data":"99c1dc818600ed799ac1eba0237c40887e417e45219b31b934ce5968de3607d8"} Sep 08 08:50:37.979845 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:37.979777 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c07b7006-ab2f-4f85-bc2b-3427edcc0923","Type":"ContainerStarted","Data":"e0aab0f9e33974fd5c299af15faeb880308f3036993b130ddd415140b9475108"} Sep 08 08:50:37.997205 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:37.997155 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-76497f5b48-g9v24" podStartSLOduration=1.833562659 podStartE2EDuration="3.997135925s" totalImagesPullingTime="2.163573266s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 08:50:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:50:37.995039332 +0000 UTC m=+184.503146100" watchObservedRunningTime="2026-09-08 08:50:37.997135925 +0000 UTC m=+184.505242694" Sep 08 08:50:38.024445 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:38.024397 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.996786117 podStartE2EDuration="3.024379872s" totalImagesPullingTime="0s" totalInitContainerRuntime="27.593755ms" isStatefulPod=false podCreationTimestamp="2026-09-08 08:50:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 08:50:38.022676782 +0000 UTC m=+184.530783553" watchObservedRunningTime="2026-09-08 08:50:38.024379872 +0000 UTC m=+184.532486640" Sep 08 08:50:41.346957 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:50:41.346922 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:51:36.347254 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:51:36.347215 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:51:36.362839 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:51:36.362806 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:51:37.239363 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:51:37.239333 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 08:52:33.883077 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:52:33.883043 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xnd5g_df2da370-7d8d-414d-b82f-3d857fadd070/console-operator/2.log" Sep 08 08:52:33.886152 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:52:33.886120 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xnd5g_df2da370-7d8d-414d-b82f-3d857fadd070/console-operator/2.log" Sep 08 08:52:33.888046 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:52:33.888021 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xn2lh_04c4d313-ed7e-4fea-bec5-3ed11857a8c4/ovn-acl-logging/0.log" Sep 08 08:52:33.891594 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:52:33.891578 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xn2lh_04c4d313-ed7e-4fea-bec5-3ed11857a8c4/ovn-acl-logging/0.log" Sep 08 08:52:33.895254 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:52:33.895238 2554 kubelet.go:1648] "Image garbage collection succeeded" Sep 08 08:57:33.913541 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:57:33.913511 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xnd5g_df2da370-7d8d-414d-b82f-3d857fadd070/console-operator/2.log" Sep 08 08:57:33.914871 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:57:33.914849 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xnd5g_df2da370-7d8d-414d-b82f-3d857fadd070/console-operator/2.log" Sep 08 08:57:33.917824 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:57:33.917795 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xn2lh_04c4d313-ed7e-4fea-bec5-3ed11857a8c4/ovn-acl-logging/0.log" Sep 08 08:57:33.919463 ip-10-0-141-177 kubenswrapper[2554]: I0908 08:57:33.919437 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xn2lh_04c4d313-ed7e-4fea-bec5-3ed11857a8c4/ovn-acl-logging/0.log" Sep 08 09:01:34.666738 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:01:34.666704 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-test-apply-s9n6l"] Sep 08 09:01:34.669892 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:01:34.669874 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-s9n6l" Sep 08 09:01:34.672229 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:01:34.672204 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Sep 08 09:01:34.672229 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:01:34.672221 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 08 09:01:34.672854 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:01:34.672839 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 08 09:01:34.672937 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:01:34.672841 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-cronjob-dockercfg-knv7z\"" Sep 08 09:01:34.677573 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:01:34.677552 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-test-apply-s9n6l"] Sep 08 09:01:34.749157 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:01:34.749130 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svq4k\" (UniqueName: \"kubernetes.io/projected/11652add-5005-469d-8e15-bb63f7bc7b56-kube-api-access-svq4k\") pod \"feast-test-apply-s9n6l\" (UID: \"11652add-5005-469d-8e15-bb63f7bc7b56\") " pod="test-ns-feast/feast-test-apply-s9n6l" Sep 08 09:01:34.850245 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:01:34.850211 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svq4k\" (UniqueName: \"kubernetes.io/projected/11652add-5005-469d-8e15-bb63f7bc7b56-kube-api-access-svq4k\") pod \"feast-test-apply-s9n6l\" (UID: \"11652add-5005-469d-8e15-bb63f7bc7b56\") " pod="test-ns-feast/feast-test-apply-s9n6l" Sep 08 09:01:34.859080 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:01:34.859053 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svq4k\" (UniqueName: \"kubernetes.io/projected/11652add-5005-469d-8e15-bb63f7bc7b56-kube-api-access-svq4k\") pod \"feast-test-apply-s9n6l\" (UID: \"11652add-5005-469d-8e15-bb63f7bc7b56\") " pod="test-ns-feast/feast-test-apply-s9n6l" Sep 08 09:01:34.987518 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:01:34.987495 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-s9n6l" Sep 08 09:01:35.256813 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:01:35.256745 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-test-apply-s9n6l"] Sep 08 09:01:35.259761 ip-10-0-141-177 kubenswrapper[2554]: W0908 09:01:35.259735 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod11652add_5005_469d_8e15_bb63f7bc7b56.slice/crio-3416ed3d9c283fd1a03aecd35803f286e5b242b25dd381a948806dd411b1519d WatchSource:0}: Error finding container 3416ed3d9c283fd1a03aecd35803f286e5b242b25dd381a948806dd411b1519d: Status 404 returned error can't find the container with id 3416ed3d9c283fd1a03aecd35803f286e5b242b25dd381a948806dd411b1519d Sep 08 09:01:35.261599 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:01:35.261579 2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 08 09:01:35.655212 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:01:35.655131 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-s9n6l" event={"ID":"11652add-5005-469d-8e15-bb63f7bc7b56","Type":"ContainerStarted","Data":"3416ed3d9c283fd1a03aecd35803f286e5b242b25dd381a948806dd411b1519d"} Sep 08 09:01:40.683898 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:01:40.683857 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-s9n6l" event={"ID":"11652add-5005-469d-8e15-bb63f7bc7b56","Type":"ContainerStarted","Data":"71fb6b421e4ab0d4947da956a90fe595abfe5e9132b00ae358c7d7b52b30b356"} Sep 08 09:01:47.714477 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:01:47.714443 2554 generic.go:356] "Generic (PLEG): container finished" podID="11652add-5005-469d-8e15-bb63f7bc7b56" containerID="71fb6b421e4ab0d4947da956a90fe595abfe5e9132b00ae358c7d7b52b30b356" exitCode=0 Sep 08 09:01:47.714834 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:01:47.714517 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-s9n6l" event={"ID":"11652add-5005-469d-8e15-bb63f7bc7b56","Type":"ContainerDied","Data":"71fb6b421e4ab0d4947da956a90fe595abfe5e9132b00ae358c7d7b52b30b356"} Sep 08 09:01:48.720279 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:01:48.720244 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-s9n6l" event={"ID":"11652add-5005-469d-8e15-bb63f7bc7b56","Type":"ContainerStarted","Data":"9673412d30adc2d88ca157c69ac0af094e7523552a6d12316374db6457fa0a0b"} Sep 08 09:01:48.736786 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:01:48.736740 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-test-apply-s9n6l" podStartSLOduration=2.803286381 podStartE2EDuration="14.736724985s" totalImagesPullingTime="4.478173487s" totalInitContainerRuntime="7.455265117s" isStatefulPod=false podCreationTimestamp="2026-09-08 09:01:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 09:01:48.735834485 +0000 UTC m=+855.243941253" watchObservedRunningTime="2026-09-08 09:01:48.736724985 +0000 UTC m=+855.244831752" Sep 08 09:01:58.759271 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:01:58.759241 2554 generic.go:356] "Generic (PLEG): container finished" podID="11652add-5005-469d-8e15-bb63f7bc7b56" containerID="9673412d30adc2d88ca157c69ac0af094e7523552a6d12316374db6457fa0a0b" exitCode=0 Sep 08 09:01:58.759646 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:01:58.759317 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-s9n6l" event={"ID":"11652add-5005-469d-8e15-bb63f7bc7b56","Type":"ContainerDied","Data":"9673412d30adc2d88ca157c69ac0af094e7523552a6d12316374db6457fa0a0b"} Sep 08 09:01:59.892047 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:01:59.892024 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-s9n6l" Sep 08 09:01:59.953246 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:01:59.953217 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-svq4k\" (UniqueName: \"kubernetes.io/projected/11652add-5005-469d-8e15-bb63f7bc7b56-kube-api-access-svq4k\") pod \"11652add-5005-469d-8e15-bb63f7bc7b56\" (UID: \"11652add-5005-469d-8e15-bb63f7bc7b56\") " Sep 08 09:01:59.955273 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:01:59.955241 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11652add-5005-469d-8e15-bb63f7bc7b56-kube-api-access-svq4k" (OuterVolumeSpecName: "kube-api-access-svq4k") pod "11652add-5005-469d-8e15-bb63f7bc7b56" (UID: "11652add-5005-469d-8e15-bb63f7bc7b56"). InnerVolumeSpecName "kube-api-access-svq4k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 09:02:00.054314 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:02:00.054291 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svq4k\" (UniqueName: \"kubernetes.io/projected/11652add-5005-469d-8e15-bb63f7bc7b56-kube-api-access-svq4k\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 09:02:00.768945 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:02:00.768918 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-s9n6l" Sep 08 09:02:00.769126 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:02:00.768916 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-s9n6l" event={"ID":"11652add-5005-469d-8e15-bb63f7bc7b56","Type":"ContainerDied","Data":"3416ed3d9c283fd1a03aecd35803f286e5b242b25dd381a948806dd411b1519d"} Sep 08 09:02:00.769126 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:02:00.769023 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3416ed3d9c283fd1a03aecd35803f286e5b242b25dd381a948806dd411b1519d" Sep 08 09:02:00.938654 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:02:00.938620 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-feast_feast-test-apply-s9n6l_11652add-5005-469d-8e15-bb63f7bc7b56/feast-0/0.log" Sep 08 09:02:00.944859 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:02:00.944835 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-feast_feast-test-apply-s9n6l_11652add-5005-469d-8e15-bb63f7bc7b56/feast-1/0.log" Sep 08 09:02:33.944461 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:02:33.944387 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xnd5g_df2da370-7d8d-414d-b82f-3d857fadd070/console-operator/2.log" Sep 08 09:02:33.944938 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:02:33.944794 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xnd5g_df2da370-7d8d-414d-b82f-3d857fadd070/console-operator/2.log" Sep 08 09:02:33.949343 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:02:33.949322 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xn2lh_04c4d313-ed7e-4fea-bec5-3ed11857a8c4/ovn-acl-logging/0.log" Sep 08 09:02:33.949638 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:02:33.949623 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xn2lh_04c4d313-ed7e-4fea-bec5-3ed11857a8c4/ovn-acl-logging/0.log" Sep 08 09:02:43.462650 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:02:43.462614 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-test-apply-s9n6l"] Sep 08 09:02:43.474626 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:02:43.474598 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-test-apply-s9n6l"] Sep 08 09:02:44.032029 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:02:44.031970 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11652add-5005-469d-8e15-bb63f7bc7b56" path="/var/lib/kubelet/pods/11652add-5005-469d-8e15-bb63f7bc7b56/volumes" Sep 08 09:07:33.972428 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:07:33.972395 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xnd5g_df2da370-7d8d-414d-b82f-3d857fadd070/console-operator/2.log" Sep 08 09:07:33.973503 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:07:33.973480 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xnd5g_df2da370-7d8d-414d-b82f-3d857fadd070/console-operator/2.log" Sep 08 09:07:33.977324 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:07:33.977300 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xn2lh_04c4d313-ed7e-4fea-bec5-3ed11857a8c4/ovn-acl-logging/0.log" Sep 08 09:07:33.978238 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:07:33.978219 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xn2lh_04c4d313-ed7e-4fea-bec5-3ed11857a8c4/ovn-acl-logging/0.log" Sep 08 09:08:22.833022 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:22.832988 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98"] Sep 08 09:08:22.833602 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:22.833480 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11652add-5005-469d-8e15-bb63f7bc7b56" containerName="feast-1" Sep 08 09:08:22.833602 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:22.833499 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="11652add-5005-469d-8e15-bb63f7bc7b56" containerName="feast-1" Sep 08 09:08:22.833602 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:22.833540 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11652add-5005-469d-8e15-bb63f7bc7b56" containerName="feast-0" Sep 08 09:08:22.833602 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:22.833550 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="11652add-5005-469d-8e15-bb63f7bc7b56" containerName="feast-0" Sep 08 09:08:22.833825 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:22.833628 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="11652add-5005-469d-8e15-bb63f7bc7b56" containerName="feast-1" Sep 08 09:08:22.836386 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:22.836365 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:08:22.838908 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:22.838889 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-fqtrt\"" Sep 08 09:08:22.839064 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:22.838888 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 08 09:08:22.839064 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:22.838890 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 08 09:08:22.839505 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:22.839488 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 08 09:08:22.839567 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:22.839510 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 08 09:08:22.839567 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:22.839490 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 08 09:08:22.839567 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:22.839540 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 08 09:08:22.849652 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:22.849632 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98"] Sep 08 09:08:22.921999 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:22.921956 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/3a8f82c3-575c-4fe7-b50f-16e834046bb6-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-z2k98\" (UID: \"3a8f82c3-575c-4fe7-b50f-16e834046bb6\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:08:22.922109 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:22.922034 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7d4f\" (UniqueName: \"kubernetes.io/projected/3a8f82c3-575c-4fe7-b50f-16e834046bb6-kube-api-access-c7d4f\") pod \"feast-simple-feast-setup-57576b8fcf-z2k98\" (UID: \"3a8f82c3-575c-4fe7-b50f-16e834046bb6\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:08:22.922109 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:22.922063 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3a8f82c3-575c-4fe7-b50f-16e834046bb6-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-z2k98\" (UID: \"3a8f82c3-575c-4fe7-b50f-16e834046bb6\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:08:22.922109 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:22.922103 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3a8f82c3-575c-4fe7-b50f-16e834046bb6-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-z2k98\" (UID: \"3a8f82c3-575c-4fe7-b50f-16e834046bb6\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:08:22.922232 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:22.922136 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3a8f82c3-575c-4fe7-b50f-16e834046bb6-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-z2k98\" (UID: \"3a8f82c3-575c-4fe7-b50f-16e834046bb6\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:08:22.922232 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:22.922174 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3a8f82c3-575c-4fe7-b50f-16e834046bb6-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-z2k98\" (UID: \"3a8f82c3-575c-4fe7-b50f-16e834046bb6\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:08:23.023275 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:23.023249 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7d4f\" (UniqueName: \"kubernetes.io/projected/3a8f82c3-575c-4fe7-b50f-16e834046bb6-kube-api-access-c7d4f\") pod \"feast-simple-feast-setup-57576b8fcf-z2k98\" (UID: \"3a8f82c3-575c-4fe7-b50f-16e834046bb6\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:08:23.023412 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:23.023280 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3a8f82c3-575c-4fe7-b50f-16e834046bb6-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-z2k98\" (UID: \"3a8f82c3-575c-4fe7-b50f-16e834046bb6\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:08:23.023412 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:23.023319 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3a8f82c3-575c-4fe7-b50f-16e834046bb6-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-z2k98\" (UID: \"3a8f82c3-575c-4fe7-b50f-16e834046bb6\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:08:23.023412 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:23.023336 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3a8f82c3-575c-4fe7-b50f-16e834046bb6-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-z2k98\" (UID: \"3a8f82c3-575c-4fe7-b50f-16e834046bb6\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:08:23.023412 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:23.023354 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3a8f82c3-575c-4fe7-b50f-16e834046bb6-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-z2k98\" (UID: \"3a8f82c3-575c-4fe7-b50f-16e834046bb6\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:08:23.023412 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:23.023383 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/3a8f82c3-575c-4fe7-b50f-16e834046bb6-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-z2k98\" (UID: \"3a8f82c3-575c-4fe7-b50f-16e834046bb6\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:08:23.023665 ip-10-0-141-177 kubenswrapper[2554]: E0908 09:08:23.023487 2554 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Sep 08 09:08:23.023665 ip-10-0-141-177 kubenswrapper[2554]: E0908 09:08:23.023554 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a8f82c3-575c-4fe7-b50f-16e834046bb6-registry-tls podName:3a8f82c3-575c-4fe7-b50f-16e834046bb6 nodeName:}" failed. No retries permitted until 2026-09-08 09:08:23.523538103 +0000 UTC m=+1250.031644850 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/3a8f82c3-575c-4fe7-b50f-16e834046bb6-registry-tls") pod "feast-simple-feast-setup-57576b8fcf-z2k98" (UID: "3a8f82c3-575c-4fe7-b50f-16e834046bb6") : secret "feast-simple-feast-setup-registry-tls" not found Sep 08 09:08:23.023793 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:23.023768 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3a8f82c3-575c-4fe7-b50f-16e834046bb6-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-z2k98\" (UID: \"3a8f82c3-575c-4fe7-b50f-16e834046bb6\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:08:23.025756 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:23.025735 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3a8f82c3-575c-4fe7-b50f-16e834046bb6-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-z2k98\" (UID: \"3a8f82c3-575c-4fe7-b50f-16e834046bb6\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:08:23.025879 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:23.025861 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3a8f82c3-575c-4fe7-b50f-16e834046bb6-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-z2k98\" (UID: \"3a8f82c3-575c-4fe7-b50f-16e834046bb6\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:08:23.025932 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:23.025861 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3a8f82c3-575c-4fe7-b50f-16e834046bb6-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-z2k98\" (UID: \"3a8f82c3-575c-4fe7-b50f-16e834046bb6\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:08:23.031501 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:23.031476 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7d4f\" (UniqueName: \"kubernetes.io/projected/3a8f82c3-575c-4fe7-b50f-16e834046bb6-kube-api-access-c7d4f\") pod \"feast-simple-feast-setup-57576b8fcf-z2k98\" (UID: \"3a8f82c3-575c-4fe7-b50f-16e834046bb6\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:08:23.526830 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:23.526798 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/3a8f82c3-575c-4fe7-b50f-16e834046bb6-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-z2k98\" (UID: \"3a8f82c3-575c-4fe7-b50f-16e834046bb6\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:08:23.529140 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:23.529119 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/3a8f82c3-575c-4fe7-b50f-16e834046bb6-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-z2k98\" (UID: \"3a8f82c3-575c-4fe7-b50f-16e834046bb6\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:08:23.755274 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:23.755227 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:08:24.033749 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:24.033730 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98"] Sep 08 09:08:24.035338 ip-10-0-141-177 kubenswrapper[2554]: W0908 09:08:24.035317 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3a8f82c3_575c_4fe7_b50f_16e834046bb6.slice/crio-eb0150c903621e0a691ae842a0d008e8292454efa63e6f1bd9fa76e801d8091e WatchSource:0}: Error finding container eb0150c903621e0a691ae842a0d008e8292454efa63e6f1bd9fa76e801d8091e: Status 404 returned error can't find the container with id eb0150c903621e0a691ae842a0d008e8292454efa63e6f1bd9fa76e801d8091e Sep 08 09:08:24.037041 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:24.037024 2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 08 09:08:24.353134 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:24.353102 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" event={"ID":"3a8f82c3-575c-4fe7-b50f-16e834046bb6","Type":"ContainerStarted","Data":"eb0150c903621e0a691ae842a0d008e8292454efa63e6f1bd9fa76e801d8091e"} Sep 08 09:08:34.003217 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:34.003133 2554 scope.go:117] "RemoveContainer" containerID="9673412d30adc2d88ca157c69ac0af094e7523552a6d12316374db6457fa0a0b" Sep 08 09:08:36.903934 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:36.903731 2554 scope.go:117] "RemoveContainer" containerID="71fb6b421e4ab0d4947da956a90fe595abfe5e9132b00ae358c7d7b52b30b356" Sep 08 09:08:37.428273 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:37.428228 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" event={"ID":"3a8f82c3-575c-4fe7-b50f-16e834046bb6","Type":"ContainerStarted","Data":"c0dee5691ebf2437622168bbef87d4992d9a77c8ce3b6c811450fb74f176c961"} Sep 08 09:08:42.451167 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:42.451132 2554 generic.go:356] "Generic (PLEG): container finished" podID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerID="c0dee5691ebf2437622168bbef87d4992d9a77c8ce3b6c811450fb74f176c961" exitCode=0 Sep 08 09:08:42.451539 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:42.451204 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" event={"ID":"3a8f82c3-575c-4fe7-b50f-16e834046bb6","Type":"ContainerDied","Data":"c0dee5691ebf2437622168bbef87d4992d9a77c8ce3b6c811450fb74f176c961"} Sep 08 09:08:43.457253 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:43.457214 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" event={"ID":"3a8f82c3-575c-4fe7-b50f-16e834046bb6","Type":"ContainerStarted","Data":"03a49c71bceda4549386d76c0660ae0550b1a99d5c99a98a659e9a2e20e23cab"} Sep 08 09:08:48.480505 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:48.480473 2554 generic.go:356] "Generic (PLEG): container finished" podID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerID="03a49c71bceda4549386d76c0660ae0550b1a99d5c99a98a659e9a2e20e23cab" exitCode=0 Sep 08 09:08:48.480912 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:48.480552 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" event={"ID":"3a8f82c3-575c-4fe7-b50f-16e834046bb6","Type":"ContainerDied","Data":"03a49c71bceda4549386d76c0660ae0550b1a99d5c99a98a659e9a2e20e23cab"} Sep 08 09:08:49.491973 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:49.491933 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" event={"ID":"3a8f82c3-575c-4fe7-b50f-16e834046bb6","Type":"ContainerStarted","Data":"e58d11544f56cbe3202cb10dda14d6a37cb8997ea8b67697be0b8b0b78c97bff"} Sep 08 09:08:49.492462 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:49.492015 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" event={"ID":"3a8f82c3-575c-4fe7-b50f-16e834046bb6","Type":"ContainerStarted","Data":"0b48ad6e062899829d834845a2fc6a002e2a5611a370452cb6f0e40c5fbe456f"} Sep 08 09:08:49.492462 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:49.492040 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" event={"ID":"3a8f82c3-575c-4fe7-b50f-16e834046bb6","Type":"ContainerStarted","Data":"dd8a41da2b8f4634e5c3b127ec444408093acfc48c8f6062d9492ccc2e00cd0e"} Sep 08 09:08:49.492462 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:49.492058 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" event={"ID":"3a8f82c3-575c-4fe7-b50f-16e834046bb6","Type":"ContainerStarted","Data":"07fd7c4f189a824f38ae6f22fd663d8cd5ea50a5f1621c5999c2ffbbaf54a74a"} Sep 08 09:08:49.518040 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:49.517953 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" podStartSLOduration=14.359591821 podStartE2EDuration="27.517930711s" totalImagesPullingTime="13.15833889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 09:08:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 09:08:49.515841499 +0000 UTC m=+1276.023948272" watchObservedRunningTime="2026-09-08 09:08:49.517930711 +0000 UTC m=+1276.026037480" Sep 08 09:08:50.756059 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:50.755869 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:08:50.756059 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:50.755929 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:08:50.756059 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:50.755948 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:08:50.756669 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:50.756118 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:08:50.758313 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:50.758257 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" podUID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.27:8443: connect: connection refused" Sep 08 09:08:50.758548 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:50.758295 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" podUID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.27:6571: connect: connection refused" Sep 08 09:08:50.758652 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:50.758314 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" podUID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerName="online" probeResult="failure" output="Get \"https://10.132.0.27:6567/health\": dial tcp 10.132.0.27:6567: connect: connection refused" Sep 08 09:08:50.758652 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:50.758335 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" podUID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.27:8816: connect: connection refused" Sep 08 09:08:53.755582 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:53.755540 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:08:53.756210 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:53.756180 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:08:53.756210 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:53.755746 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" podUID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.27:8443: connect: connection refused" Sep 08 09:08:53.756484 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:53.755806 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" podUID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerName="online" probeResult="failure" output="Get \"https://10.132.0.27:6567/health\": dial tcp 10.132.0.27:6567: connect: connection refused" Sep 08 09:08:53.756484 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:53.755911 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" podUID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.27:8816: connect: connection refused" Sep 08 09:08:53.756640 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:53.756215 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:08:53.756640 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:53.755781 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" podUID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.27:6571: connect: connection refused" Sep 08 09:08:53.756640 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:53.756629 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:08:56.755685 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:56.755643 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" podUID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.27:8443: connect: connection refused" Sep 08 09:08:56.756674 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:56.756651 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:08:56.756853 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:56.756780 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:08:56.757526 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:56.757504 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:08:56.757660 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:56.757643 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:08:56.761674 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:56.761656 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:08:57.541610 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:57.541574 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:08:59.756640 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:08:59.756610 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:09:00.550776 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:09:00.550745 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:10:03.013037 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:03.012938 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98"] Sep 08 09:10:03.013597 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:03.013257 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" podUID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerName="registry" containerID="cri-o://07fd7c4f189a824f38ae6f22fd663d8cd5ea50a5f1621c5999c2ffbbaf54a74a" gracePeriod=30 Sep 08 09:10:03.013597 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:03.013368 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" podUID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerName="offline" containerID="cri-o://0b48ad6e062899829d834845a2fc6a002e2a5611a370452cb6f0e40c5fbe456f" gracePeriod=30 Sep 08 09:10:03.013597 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:03.013418 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" podUID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerName="online" containerID="cri-o://dd8a41da2b8f4634e5c3b127ec444408093acfc48c8f6062d9492ccc2e00cd0e" gracePeriod=30 Sep 08 09:10:03.013597 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:03.013381 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" podUID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerName="ui" containerID="cri-o://e58d11544f56cbe3202cb10dda14d6a37cb8997ea8b67697be0b8b0b78c97bff" gracePeriod=30 Sep 08 09:10:03.441952 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:03.441851 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc"] Sep 08 09:10:03.445874 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:03.445851 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:10:03.448022 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:03.447993 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-d7jfx\"" Sep 08 09:10:03.456421 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:03.456394 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc"] Sep 08 09:10:03.568357 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:03.568309 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9a288e10-6517-4367-8f4d-9621b23a01cf-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-8wzvc\" (UID: \"9a288e10-6517-4367-8f4d-9621b23a01cf\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:10:03.568577 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:03.568417 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9a288e10-6517-4367-8f4d-9621b23a01cf-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-8wzvc\" (UID: \"9a288e10-6517-4367-8f4d-9621b23a01cf\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:10:03.568577 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:03.568449 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn7k7\" (UniqueName: \"kubernetes.io/projected/9a288e10-6517-4367-8f4d-9621b23a01cf-kube-api-access-dn7k7\") pod \"feast-simple-feast-setup-57576b8fcf-8wzvc\" (UID: \"9a288e10-6517-4367-8f4d-9621b23a01cf\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:10:03.568577 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:03.568485 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9a288e10-6517-4367-8f4d-9621b23a01cf-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-8wzvc\" (UID: \"9a288e10-6517-4367-8f4d-9621b23a01cf\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:10:03.568577 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:03.568566 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/9a288e10-6517-4367-8f4d-9621b23a01cf-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-8wzvc\" (UID: \"9a288e10-6517-4367-8f4d-9621b23a01cf\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:10:03.568781 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:03.568603 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9a288e10-6517-4367-8f4d-9621b23a01cf-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-8wzvc\" (UID: \"9a288e10-6517-4367-8f4d-9621b23a01cf\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:10:03.669361 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:03.669324 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9a288e10-6517-4367-8f4d-9621b23a01cf-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-8wzvc\" (UID: \"9a288e10-6517-4367-8f4d-9621b23a01cf\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:10:03.669523 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:03.669367 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn7k7\" (UniqueName: \"kubernetes.io/projected/9a288e10-6517-4367-8f4d-9621b23a01cf-kube-api-access-dn7k7\") pod \"feast-simple-feast-setup-57576b8fcf-8wzvc\" (UID: \"9a288e10-6517-4367-8f4d-9621b23a01cf\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:10:03.669523 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:03.669394 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9a288e10-6517-4367-8f4d-9621b23a01cf-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-8wzvc\" (UID: \"9a288e10-6517-4367-8f4d-9621b23a01cf\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:10:03.669523 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:03.669429 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/9a288e10-6517-4367-8f4d-9621b23a01cf-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-8wzvc\" (UID: \"9a288e10-6517-4367-8f4d-9621b23a01cf\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:10:03.669523 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:03.669453 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9a288e10-6517-4367-8f4d-9621b23a01cf-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-8wzvc\" (UID: \"9a288e10-6517-4367-8f4d-9621b23a01cf\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:10:03.669719 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:03.669606 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9a288e10-6517-4367-8f4d-9621b23a01cf-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-8wzvc\" (UID: \"9a288e10-6517-4367-8f4d-9621b23a01cf\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:10:03.669797 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:03.669781 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9a288e10-6517-4367-8f4d-9621b23a01cf-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-8wzvc\" (UID: \"9a288e10-6517-4367-8f4d-9621b23a01cf\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:10:03.672602 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:03.672571 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9a288e10-6517-4367-8f4d-9621b23a01cf-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-8wzvc\" (UID: \"9a288e10-6517-4367-8f4d-9621b23a01cf\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:10:03.672717 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:03.672610 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/9a288e10-6517-4367-8f4d-9621b23a01cf-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-8wzvc\" (UID: \"9a288e10-6517-4367-8f4d-9621b23a01cf\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:10:03.672717 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:03.672642 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9a288e10-6517-4367-8f4d-9621b23a01cf-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-8wzvc\" (UID: \"9a288e10-6517-4367-8f4d-9621b23a01cf\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:10:03.673096 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:03.673078 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9a288e10-6517-4367-8f4d-9621b23a01cf-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-8wzvc\" (UID: \"9a288e10-6517-4367-8f4d-9621b23a01cf\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:10:03.677600 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:03.677577 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn7k7\" (UniqueName: \"kubernetes.io/projected/9a288e10-6517-4367-8f4d-9621b23a01cf-kube-api-access-dn7k7\") pod \"feast-simple-feast-setup-57576b8fcf-8wzvc\" (UID: \"9a288e10-6517-4367-8f4d-9621b23a01cf\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:10:03.758113 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:03.758072 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:10:03.826393 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:03.826335 2554 generic.go:356] "Generic (PLEG): container finished" podID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerID="e58d11544f56cbe3202cb10dda14d6a37cb8997ea8b67697be0b8b0b78c97bff" exitCode=0 Sep 08 09:10:03.826393 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:03.826385 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" event={"ID":"3a8f82c3-575c-4fe7-b50f-16e834046bb6","Type":"ContainerDied","Data":"e58d11544f56cbe3202cb10dda14d6a37cb8997ea8b67697be0b8b0b78c97bff"} Sep 08 09:10:04.038645 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:04.038564 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc"] Sep 08 09:10:04.041827 ip-10-0-141-177 kubenswrapper[2554]: W0908 09:10:04.041802 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a288e10_6517_4367_8f4d_9621b23a01cf.slice/crio-1725f6ba60c346971836810801cd68fe3cfd3bb143078d489e100e33cae21b6e WatchSource:0}: Error finding container 1725f6ba60c346971836810801cd68fe3cfd3bb143078d489e100e33cae21b6e: Status 404 returned error can't find the container with id 1725f6ba60c346971836810801cd68fe3cfd3bb143078d489e100e33cae21b6e Sep 08 09:10:04.831773 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:04.831736 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" event={"ID":"9a288e10-6517-4367-8f4d-9621b23a01cf","Type":"ContainerStarted","Data":"dffaa2341a0bb67c0e56f1194ae924c06143e83c4fc624dd9404d56a2c8c3449"} Sep 08 09:10:04.831952 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:04.831793 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" event={"ID":"9a288e10-6517-4367-8f4d-9621b23a01cf","Type":"ContainerStarted","Data":"1725f6ba60c346971836810801cd68fe3cfd3bb143078d489e100e33cae21b6e"} Sep 08 09:10:04.835103 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:04.835074 2554 generic.go:356] "Generic (PLEG): container finished" podID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerID="dd8a41da2b8f4634e5c3b127ec444408093acfc48c8f6062d9492ccc2e00cd0e" exitCode=0 Sep 08 09:10:04.835207 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:04.835110 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" event={"ID":"3a8f82c3-575c-4fe7-b50f-16e834046bb6","Type":"ContainerDied","Data":"dd8a41da2b8f4634e5c3b127ec444408093acfc48c8f6062d9492ccc2e00cd0e"} Sep 08 09:10:07.539350 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:07.539308 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" podUID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerName="online" probeResult="failure" output="Get \"https://10.132.0.27:6567/health\": dial tcp 10.132.0.27:6567: connect: connection refused" Sep 08 09:10:09.859929 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:09.859896 2554 generic.go:356] "Generic (PLEG): container finished" podID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerID="dffaa2341a0bb67c0e56f1194ae924c06143e83c4fc624dd9404d56a2c8c3449" exitCode=0 Sep 08 09:10:09.860327 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:09.859969 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" event={"ID":"9a288e10-6517-4367-8f4d-9621b23a01cf","Type":"ContainerDied","Data":"dffaa2341a0bb67c0e56f1194ae924c06143e83c4fc624dd9404d56a2c8c3449"} Sep 08 09:10:10.551382 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:10.551331 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" podUID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.27:8443: connect: connection refused" Sep 08 09:10:10.866169 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:10.866087 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" event={"ID":"9a288e10-6517-4367-8f4d-9621b23a01cf","Type":"ContainerStarted","Data":"a56ad026927d1cff04f97601439843dc714cc899a85450bd81bff2bb3e69eba8"} Sep 08 09:10:15.889363 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:15.889283 2554 generic.go:356] "Generic (PLEG): container finished" podID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerID="a56ad026927d1cff04f97601439843dc714cc899a85450bd81bff2bb3e69eba8" exitCode=0 Sep 08 09:10:15.889363 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:15.889351 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" event={"ID":"9a288e10-6517-4367-8f4d-9621b23a01cf","Type":"ContainerDied","Data":"a56ad026927d1cff04f97601439843dc714cc899a85450bd81bff2bb3e69eba8"} Sep 08 09:10:16.900990 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:16.900921 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" event={"ID":"9a288e10-6517-4367-8f4d-9621b23a01cf","Type":"ContainerStarted","Data":"0d9b5e32057bf82af5170442cba560367897187748a2172aef5b83bda7966924"} Sep 08 09:10:16.901470 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:16.901022 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" event={"ID":"9a288e10-6517-4367-8f4d-9621b23a01cf","Type":"ContainerStarted","Data":"5282045ed519618fa730d2c026e2072ba8d0b70a61529ef83ede0c5eeebeff4d"} Sep 08 09:10:16.901470 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:16.901047 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" event={"ID":"9a288e10-6517-4367-8f4d-9621b23a01cf","Type":"ContainerStarted","Data":"512480e0e02a9d5bbf671938894d3e2a95cbf0be52b6439019ada3ee64c7ea44"} Sep 08 09:10:16.901470 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:16.901087 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" event={"ID":"9a288e10-6517-4367-8f4d-9621b23a01cf","Type":"ContainerStarted","Data":"cb1f53fa0e646b669833afdbf93c1d34e2c6c4ba6f1f5d3f8434f499cd6f9c7d"} Sep 08 09:10:16.930630 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:16.930562 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" podStartSLOduration=13.930540757 podStartE2EDuration="13.930540757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 09:10:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 09:10:16.928239263 +0000 UTC m=+1363.436346034" watchObservedRunningTime="2026-09-08 09:10:16.930540757 +0000 UTC m=+1363.438647530" Sep 08 09:10:17.539469 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:17.539421 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" podUID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerName="online" probeResult="failure" output="Get \"https://10.132.0.27:6567/health\": dial tcp 10.132.0.27:6567: connect: connection refused" Sep 08 09:10:18.758915 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:18.758797 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:10:18.758915 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:18.758867 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:10:18.758915 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:18.758891 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:10:18.759537 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:18.759017 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:10:18.761339 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:18.761220 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" podUID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerName="online" probeResult="failure" output="Get \"https://10.132.0.28:6567/health\": dial tcp 10.132.0.28:6567: connect: connection refused" Sep 08 09:10:18.761339 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:18.761240 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" podUID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.28:8816: connect: connection refused" Sep 08 09:10:18.761339 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:18.761220 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" podUID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.28:6571: connect: connection refused" Sep 08 09:10:18.761585 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:18.761540 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" podUID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.28:8443: connect: connection refused" Sep 08 09:10:20.550415 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:20.550363 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" podUID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.27:8443: connect: connection refused" Sep 08 09:10:21.758681 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:21.758635 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" podUID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.28:6571: connect: connection refused" Sep 08 09:10:21.759188 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:21.758635 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" podUID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.28:8443: connect: connection refused" Sep 08 09:10:21.759188 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:21.758760 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" podUID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerName="online" probeResult="failure" output="Get \"https://10.132.0.28:6567/health\": dial tcp 10.132.0.28:6567: connect: connection refused" Sep 08 09:10:21.759188 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:21.759116 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" podUID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.28:8816: connect: connection refused" Sep 08 09:10:23.759080 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:23.759034 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:10:23.759541 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:23.759104 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:10:23.759541 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:23.759125 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:10:23.759541 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:23.759142 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:10:24.759724 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:24.759685 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:10:24.760122 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:24.759915 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:10:24.760122 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:24.760077 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:10:24.760767 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:24.760743 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:10:24.760882 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:24.760810 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:10:24.760947 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:24.760922 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:10:24.765053 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:24.765037 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:10:24.954569 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:24.954537 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:10:27.538849 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:27.538803 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" podUID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerName="online" probeResult="failure" output="Get \"https://10.132.0.27:6567/health\": dial tcp 10.132.0.27:6567: connect: connection refused" Sep 08 09:10:27.539283 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:27.538942 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:10:30.550785 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:30.550743 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" podUID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.27:8443: connect: connection refused" Sep 08 09:10:30.551270 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:30.550855 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:10:33.661721 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:33.661698 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:10:33.766691 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:33.766646 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3a8f82c3-575c-4fe7-b50f-16e834046bb6-online-tls\") pod \"3a8f82c3-575c-4fe7-b50f-16e834046bb6\" (UID: \"3a8f82c3-575c-4fe7-b50f-16e834046bb6\") " Sep 08 09:10:33.766872 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:33.766727 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3a8f82c3-575c-4fe7-b50f-16e834046bb6-feast-data\") pod \"3a8f82c3-575c-4fe7-b50f-16e834046bb6\" (UID: \"3a8f82c3-575c-4fe7-b50f-16e834046bb6\") " Sep 08 09:10:33.766872 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:33.766777 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/3a8f82c3-575c-4fe7-b50f-16e834046bb6-registry-tls\") pod \"3a8f82c3-575c-4fe7-b50f-16e834046bb6\" (UID: \"3a8f82c3-575c-4fe7-b50f-16e834046bb6\") " Sep 08 09:10:33.766872 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:33.766813 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c7d4f\" (UniqueName: \"kubernetes.io/projected/3a8f82c3-575c-4fe7-b50f-16e834046bb6-kube-api-access-c7d4f\") pod \"3a8f82c3-575c-4fe7-b50f-16e834046bb6\" (UID: \"3a8f82c3-575c-4fe7-b50f-16e834046bb6\") " Sep 08 09:10:33.766872 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:33.766849 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3a8f82c3-575c-4fe7-b50f-16e834046bb6-ui-tls\") pod \"3a8f82c3-575c-4fe7-b50f-16e834046bb6\" (UID: \"3a8f82c3-575c-4fe7-b50f-16e834046bb6\") " Sep 08 09:10:33.766872 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:33.766867 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3a8f82c3-575c-4fe7-b50f-16e834046bb6-offline-tls\") pod \"3a8f82c3-575c-4fe7-b50f-16e834046bb6\" (UID: \"3a8f82c3-575c-4fe7-b50f-16e834046bb6\") " Sep 08 09:10:33.767371 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:33.767336 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a8f82c3-575c-4fe7-b50f-16e834046bb6-feast-data" (OuterVolumeSpecName: "feast-data") pod "3a8f82c3-575c-4fe7-b50f-16e834046bb6" (UID: "3a8f82c3-575c-4fe7-b50f-16e834046bb6"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 09:10:33.769029 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:33.768973 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a8f82c3-575c-4fe7-b50f-16e834046bb6-online-tls" (OuterVolumeSpecName: "online-tls") pod "3a8f82c3-575c-4fe7-b50f-16e834046bb6" (UID: "3a8f82c3-575c-4fe7-b50f-16e834046bb6"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 09:10:33.769146 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:33.769046 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a8f82c3-575c-4fe7-b50f-16e834046bb6-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "3a8f82c3-575c-4fe7-b50f-16e834046bb6" (UID: "3a8f82c3-575c-4fe7-b50f-16e834046bb6"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 09:10:33.769146 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:33.769100 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a8f82c3-575c-4fe7-b50f-16e834046bb6-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "3a8f82c3-575c-4fe7-b50f-16e834046bb6" (UID: "3a8f82c3-575c-4fe7-b50f-16e834046bb6"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 09:10:33.769286 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:33.769268 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a8f82c3-575c-4fe7-b50f-16e834046bb6-kube-api-access-c7d4f" (OuterVolumeSpecName: "kube-api-access-c7d4f") pod "3a8f82c3-575c-4fe7-b50f-16e834046bb6" (UID: "3a8f82c3-575c-4fe7-b50f-16e834046bb6"). InnerVolumeSpecName "kube-api-access-c7d4f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 09:10:33.769419 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:33.769397 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a8f82c3-575c-4fe7-b50f-16e834046bb6-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "3a8f82c3-575c-4fe7-b50f-16e834046bb6" (UID: "3a8f82c3-575c-4fe7-b50f-16e834046bb6"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 09:10:33.868047 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:33.867957 2554 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3a8f82c3-575c-4fe7-b50f-16e834046bb6-online-tls\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 09:10:33.868047 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:33.868000 2554 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3a8f82c3-575c-4fe7-b50f-16e834046bb6-feast-data\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 09:10:33.868047 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:33.868010 2554 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/3a8f82c3-575c-4fe7-b50f-16e834046bb6-registry-tls\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 09:10:33.868047 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:33.868019 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7d4f\" (UniqueName: \"kubernetes.io/projected/3a8f82c3-575c-4fe7-b50f-16e834046bb6-kube-api-access-c7d4f\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 09:10:33.868047 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:33.868028 2554 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3a8f82c3-575c-4fe7-b50f-16e834046bb6-ui-tls\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 09:10:33.868047 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:33.868036 2554 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3a8f82c3-575c-4fe7-b50f-16e834046bb6-offline-tls\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 09:10:33.994733 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:33.994698 2554 generic.go:356] "Generic (PLEG): container finished" podID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerID="0b48ad6e062899829d834845a2fc6a002e2a5611a370452cb6f0e40c5fbe456f" exitCode=137 Sep 08 09:10:33.994733 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:33.994724 2554 generic.go:356] "Generic (PLEG): container finished" podID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerID="07fd7c4f189a824f38ae6f22fd663d8cd5ea50a5f1621c5999c2ffbbaf54a74a" exitCode=137 Sep 08 09:10:33.994945 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:33.994757 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" Sep 08 09:10:33.994945 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:33.994787 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" event={"ID":"3a8f82c3-575c-4fe7-b50f-16e834046bb6","Type":"ContainerDied","Data":"0b48ad6e062899829d834845a2fc6a002e2a5611a370452cb6f0e40c5fbe456f"} Sep 08 09:10:33.994945 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:33.994845 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" event={"ID":"3a8f82c3-575c-4fe7-b50f-16e834046bb6","Type":"ContainerDied","Data":"07fd7c4f189a824f38ae6f22fd663d8cd5ea50a5f1621c5999c2ffbbaf54a74a"} Sep 08 09:10:33.994945 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:33.994861 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98" event={"ID":"3a8f82c3-575c-4fe7-b50f-16e834046bb6","Type":"ContainerDied","Data":"eb0150c903621e0a691ae842a0d008e8292454efa63e6f1bd9fa76e801d8091e"} Sep 08 09:10:33.994945 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:33.994880 2554 scope.go:117] "RemoveContainer" containerID="e58d11544f56cbe3202cb10dda14d6a37cb8997ea8b67697be0b8b0b78c97bff" Sep 08 09:10:34.003596 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:34.003578 2554 scope.go:117] "RemoveContainer" containerID="0b48ad6e062899829d834845a2fc6a002e2a5611a370452cb6f0e40c5fbe456f" Sep 08 09:10:34.011176 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:34.011158 2554 scope.go:117] "RemoveContainer" containerID="dd8a41da2b8f4634e5c3b127ec444408093acfc48c8f6062d9492ccc2e00cd0e" Sep 08 09:10:34.016649 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:34.016623 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98"] Sep 08 09:10:34.020064 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:34.019806 2554 scope.go:117] "RemoveContainer" containerID="07fd7c4f189a824f38ae6f22fd663d8cd5ea50a5f1621c5999c2ffbbaf54a74a" Sep 08 09:10:34.020833 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:34.020774 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-z2k98"] Sep 08 09:10:34.030812 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:34.030797 2554 scope.go:117] "RemoveContainer" containerID="03a49c71bceda4549386d76c0660ae0550b1a99d5c99a98a659e9a2e20e23cab" Sep 08 09:10:34.035208 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:34.035183 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" path="/var/lib/kubelet/pods/3a8f82c3-575c-4fe7-b50f-16e834046bb6/volumes" Sep 08 09:10:34.040217 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:34.040198 2554 scope.go:117] "RemoveContainer" containerID="c0dee5691ebf2437622168bbef87d4992d9a77c8ce3b6c811450fb74f176c961" Sep 08 09:10:34.047540 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:34.047524 2554 scope.go:117] "RemoveContainer" containerID="e58d11544f56cbe3202cb10dda14d6a37cb8997ea8b67697be0b8b0b78c97bff" Sep 08 09:10:34.047807 ip-10-0-141-177 kubenswrapper[2554]: E0908 09:10:34.047788 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e58d11544f56cbe3202cb10dda14d6a37cb8997ea8b67697be0b8b0b78c97bff\": container with ID starting with e58d11544f56cbe3202cb10dda14d6a37cb8997ea8b67697be0b8b0b78c97bff not found: ID does not exist" containerID="e58d11544f56cbe3202cb10dda14d6a37cb8997ea8b67697be0b8b0b78c97bff" Sep 08 09:10:34.047868 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:34.047816 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e58d11544f56cbe3202cb10dda14d6a37cb8997ea8b67697be0b8b0b78c97bff"} err="failed to get container status \"e58d11544f56cbe3202cb10dda14d6a37cb8997ea8b67697be0b8b0b78c97bff\": rpc error: code = NotFound desc = could not find container \"e58d11544f56cbe3202cb10dda14d6a37cb8997ea8b67697be0b8b0b78c97bff\": container with ID starting with e58d11544f56cbe3202cb10dda14d6a37cb8997ea8b67697be0b8b0b78c97bff not found: ID does not exist" Sep 08 09:10:34.047868 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:34.047839 2554 scope.go:117] "RemoveContainer" containerID="0b48ad6e062899829d834845a2fc6a002e2a5611a370452cb6f0e40c5fbe456f" Sep 08 09:10:34.048121 ip-10-0-141-177 kubenswrapper[2554]: E0908 09:10:34.048102 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b48ad6e062899829d834845a2fc6a002e2a5611a370452cb6f0e40c5fbe456f\": container with ID starting with 0b48ad6e062899829d834845a2fc6a002e2a5611a370452cb6f0e40c5fbe456f not found: ID does not exist" containerID="0b48ad6e062899829d834845a2fc6a002e2a5611a370452cb6f0e40c5fbe456f" Sep 08 09:10:34.048198 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:34.048129 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b48ad6e062899829d834845a2fc6a002e2a5611a370452cb6f0e40c5fbe456f"} err="failed to get container status \"0b48ad6e062899829d834845a2fc6a002e2a5611a370452cb6f0e40c5fbe456f\": rpc error: code = NotFound desc = could not find container \"0b48ad6e062899829d834845a2fc6a002e2a5611a370452cb6f0e40c5fbe456f\": container with ID starting with 0b48ad6e062899829d834845a2fc6a002e2a5611a370452cb6f0e40c5fbe456f not found: ID does not exist" Sep 08 09:10:34.048198 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:34.048152 2554 scope.go:117] "RemoveContainer" containerID="dd8a41da2b8f4634e5c3b127ec444408093acfc48c8f6062d9492ccc2e00cd0e" Sep 08 09:10:34.048452 ip-10-0-141-177 kubenswrapper[2554]: E0908 09:10:34.048431 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd8a41da2b8f4634e5c3b127ec444408093acfc48c8f6062d9492ccc2e00cd0e\": container with ID starting with dd8a41da2b8f4634e5c3b127ec444408093acfc48c8f6062d9492ccc2e00cd0e not found: ID does not exist" containerID="dd8a41da2b8f4634e5c3b127ec444408093acfc48c8f6062d9492ccc2e00cd0e" Sep 08 09:10:34.048491 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:34.048458 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd8a41da2b8f4634e5c3b127ec444408093acfc48c8f6062d9492ccc2e00cd0e"} err="failed to get container status \"dd8a41da2b8f4634e5c3b127ec444408093acfc48c8f6062d9492ccc2e00cd0e\": rpc error: code = NotFound desc = could not find container \"dd8a41da2b8f4634e5c3b127ec444408093acfc48c8f6062d9492ccc2e00cd0e\": container with ID starting with dd8a41da2b8f4634e5c3b127ec444408093acfc48c8f6062d9492ccc2e00cd0e not found: ID does not exist" Sep 08 09:10:34.048491 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:34.048472 2554 scope.go:117] "RemoveContainer" containerID="07fd7c4f189a824f38ae6f22fd663d8cd5ea50a5f1621c5999c2ffbbaf54a74a" Sep 08 09:10:34.048713 ip-10-0-141-177 kubenswrapper[2554]: E0908 09:10:34.048694 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07fd7c4f189a824f38ae6f22fd663d8cd5ea50a5f1621c5999c2ffbbaf54a74a\": container with ID starting with 07fd7c4f189a824f38ae6f22fd663d8cd5ea50a5f1621c5999c2ffbbaf54a74a not found: ID does not exist" containerID="07fd7c4f189a824f38ae6f22fd663d8cd5ea50a5f1621c5999c2ffbbaf54a74a" Sep 08 09:10:34.048755 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:34.048717 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07fd7c4f189a824f38ae6f22fd663d8cd5ea50a5f1621c5999c2ffbbaf54a74a"} err="failed to get container status \"07fd7c4f189a824f38ae6f22fd663d8cd5ea50a5f1621c5999c2ffbbaf54a74a\": rpc error: code = NotFound desc = could not find container \"07fd7c4f189a824f38ae6f22fd663d8cd5ea50a5f1621c5999c2ffbbaf54a74a\": container with ID starting with 07fd7c4f189a824f38ae6f22fd663d8cd5ea50a5f1621c5999c2ffbbaf54a74a not found: ID does not exist" Sep 08 09:10:34.048755 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:34.048730 2554 scope.go:117] "RemoveContainer" containerID="03a49c71bceda4549386d76c0660ae0550b1a99d5c99a98a659e9a2e20e23cab" Sep 08 09:10:34.049015 ip-10-0-141-177 kubenswrapper[2554]: E0908 09:10:34.048971 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03a49c71bceda4549386d76c0660ae0550b1a99d5c99a98a659e9a2e20e23cab\": container with ID starting with 03a49c71bceda4549386d76c0660ae0550b1a99d5c99a98a659e9a2e20e23cab not found: ID does not exist" containerID="03a49c71bceda4549386d76c0660ae0550b1a99d5c99a98a659e9a2e20e23cab" Sep 08 09:10:34.049084 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:34.049022 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03a49c71bceda4549386d76c0660ae0550b1a99d5c99a98a659e9a2e20e23cab"} err="failed to get container status \"03a49c71bceda4549386d76c0660ae0550b1a99d5c99a98a659e9a2e20e23cab\": rpc error: code = NotFound desc = could not find container \"03a49c71bceda4549386d76c0660ae0550b1a99d5c99a98a659e9a2e20e23cab\": container with ID starting with 03a49c71bceda4549386d76c0660ae0550b1a99d5c99a98a659e9a2e20e23cab not found: ID does not exist" Sep 08 09:10:34.049084 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:34.049041 2554 scope.go:117] "RemoveContainer" containerID="c0dee5691ebf2437622168bbef87d4992d9a77c8ce3b6c811450fb74f176c961" Sep 08 09:10:34.049277 ip-10-0-141-177 kubenswrapper[2554]: E0908 09:10:34.049261 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0dee5691ebf2437622168bbef87d4992d9a77c8ce3b6c811450fb74f176c961\": container with ID starting with c0dee5691ebf2437622168bbef87d4992d9a77c8ce3b6c811450fb74f176c961 not found: ID does not exist" containerID="c0dee5691ebf2437622168bbef87d4992d9a77c8ce3b6c811450fb74f176c961" Sep 08 09:10:34.049321 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:34.049282 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0dee5691ebf2437622168bbef87d4992d9a77c8ce3b6c811450fb74f176c961"} err="failed to get container status \"c0dee5691ebf2437622168bbef87d4992d9a77c8ce3b6c811450fb74f176c961\": rpc error: code = NotFound desc = could not find container \"c0dee5691ebf2437622168bbef87d4992d9a77c8ce3b6c811450fb74f176c961\": container with ID starting with c0dee5691ebf2437622168bbef87d4992d9a77c8ce3b6c811450fb74f176c961 not found: ID does not exist" Sep 08 09:10:34.049321 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:34.049294 2554 scope.go:117] "RemoveContainer" containerID="e58d11544f56cbe3202cb10dda14d6a37cb8997ea8b67697be0b8b0b78c97bff" Sep 08 09:10:34.049490 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:34.049474 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e58d11544f56cbe3202cb10dda14d6a37cb8997ea8b67697be0b8b0b78c97bff"} err="failed to get container status \"e58d11544f56cbe3202cb10dda14d6a37cb8997ea8b67697be0b8b0b78c97bff\": rpc error: code = NotFound desc = could not find container \"e58d11544f56cbe3202cb10dda14d6a37cb8997ea8b67697be0b8b0b78c97bff\": container with ID starting with e58d11544f56cbe3202cb10dda14d6a37cb8997ea8b67697be0b8b0b78c97bff not found: ID does not exist" Sep 08 09:10:34.049537 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:34.049491 2554 scope.go:117] "RemoveContainer" containerID="0b48ad6e062899829d834845a2fc6a002e2a5611a370452cb6f0e40c5fbe456f" Sep 08 09:10:34.049714 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:34.049697 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b48ad6e062899829d834845a2fc6a002e2a5611a370452cb6f0e40c5fbe456f"} err="failed to get container status \"0b48ad6e062899829d834845a2fc6a002e2a5611a370452cb6f0e40c5fbe456f\": rpc error: code = NotFound desc = could not find container \"0b48ad6e062899829d834845a2fc6a002e2a5611a370452cb6f0e40c5fbe456f\": container with ID starting with 0b48ad6e062899829d834845a2fc6a002e2a5611a370452cb6f0e40c5fbe456f not found: ID does not exist" Sep 08 09:10:34.049765 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:34.049715 2554 scope.go:117] "RemoveContainer" containerID="dd8a41da2b8f4634e5c3b127ec444408093acfc48c8f6062d9492ccc2e00cd0e" Sep 08 09:10:34.049906 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:34.049886 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd8a41da2b8f4634e5c3b127ec444408093acfc48c8f6062d9492ccc2e00cd0e"} err="failed to get container status \"dd8a41da2b8f4634e5c3b127ec444408093acfc48c8f6062d9492ccc2e00cd0e\": rpc error: code = NotFound desc = could not find container \"dd8a41da2b8f4634e5c3b127ec444408093acfc48c8f6062d9492ccc2e00cd0e\": container with ID starting with dd8a41da2b8f4634e5c3b127ec444408093acfc48c8f6062d9492ccc2e00cd0e not found: ID does not exist" Sep 08 09:10:34.049943 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:34.049908 2554 scope.go:117] "RemoveContainer" containerID="07fd7c4f189a824f38ae6f22fd663d8cd5ea50a5f1621c5999c2ffbbaf54a74a" Sep 08 09:10:34.050124 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:34.050107 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07fd7c4f189a824f38ae6f22fd663d8cd5ea50a5f1621c5999c2ffbbaf54a74a"} err="failed to get container status \"07fd7c4f189a824f38ae6f22fd663d8cd5ea50a5f1621c5999c2ffbbaf54a74a\": rpc error: code = NotFound desc = could not find container \"07fd7c4f189a824f38ae6f22fd663d8cd5ea50a5f1621c5999c2ffbbaf54a74a\": container with ID starting with 07fd7c4f189a824f38ae6f22fd663d8cd5ea50a5f1621c5999c2ffbbaf54a74a not found: ID does not exist" Sep 08 09:10:34.050124 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:34.050123 2554 scope.go:117] "RemoveContainer" containerID="03a49c71bceda4549386d76c0660ae0550b1a99d5c99a98a659e9a2e20e23cab" Sep 08 09:10:34.050309 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:34.050295 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03a49c71bceda4549386d76c0660ae0550b1a99d5c99a98a659e9a2e20e23cab"} err="failed to get container status \"03a49c71bceda4549386d76c0660ae0550b1a99d5c99a98a659e9a2e20e23cab\": rpc error: code = NotFound desc = could not find container \"03a49c71bceda4549386d76c0660ae0550b1a99d5c99a98a659e9a2e20e23cab\": container with ID starting with 03a49c71bceda4549386d76c0660ae0550b1a99d5c99a98a659e9a2e20e23cab not found: ID does not exist" Sep 08 09:10:34.050349 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:34.050309 2554 scope.go:117] "RemoveContainer" containerID="c0dee5691ebf2437622168bbef87d4992d9a77c8ce3b6c811450fb74f176c961" Sep 08 09:10:34.050510 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:10:34.050492 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0dee5691ebf2437622168bbef87d4992d9a77c8ce3b6c811450fb74f176c961"} err="failed to get container status \"c0dee5691ebf2437622168bbef87d4992d9a77c8ce3b6c811450fb74f176c961\": rpc error: code = NotFound desc = could not find container \"c0dee5691ebf2437622168bbef87d4992d9a77c8ce3b6c811450fb74f176c961\": container with ID starting with c0dee5691ebf2437622168bbef87d4992d9a77c8ce3b6c811450fb74f176c961 not found: ID does not exist" Sep 08 09:11:28.426122 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.426044 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj"] Sep 08 09:11:28.426501 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.426372 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerName="registry" Sep 08 09:11:28.426501 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.426383 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerName="registry" Sep 08 09:11:28.426501 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.426398 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerName="online" Sep 08 09:11:28.426501 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.426404 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerName="online" Sep 08 09:11:28.426501 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.426411 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerName="ui" Sep 08 09:11:28.426501 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.426417 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerName="ui" Sep 08 09:11:28.426501 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.426424 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerName="feast-init" Sep 08 09:11:28.426501 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.426429 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerName="feast-init" Sep 08 09:11:28.426501 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.426438 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerName="feast-apply" Sep 08 09:11:28.426501 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.426443 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerName="feast-apply" Sep 08 09:11:28.426501 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.426449 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerName="offline" Sep 08 09:11:28.426501 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.426453 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerName="offline" Sep 08 09:11:28.427016 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.426512 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerName="ui" Sep 08 09:11:28.427016 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.426522 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerName="offline" Sep 08 09:11:28.427016 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.426528 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerName="online" Sep 08 09:11:28.427016 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.426536 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a8f82c3-575c-4fe7-b50f-16e834046bb6" containerName="registry" Sep 08 09:11:28.430209 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.430183 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" Sep 08 09:11:28.433016 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.432968 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 08 09:11:28.433155 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.433107 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 08 09:11:28.433155 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.433110 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 08 09:11:28.433282 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.433206 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 08 09:11:28.433282 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.433265 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 08 09:11:28.433787 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.433768 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-8zbp7\"" Sep 08 09:11:28.433895 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.433851 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 08 09:11:28.444152 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.444133 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj"] Sep 08 09:11:28.529931 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.529898 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9af7eca1-1b66-4af7-a603-41c15bdd804b-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-xpnfj\" (UID: \"9af7eca1-1b66-4af7-a603-41c15bdd804b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" Sep 08 09:11:28.529931 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.529931 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9af7eca1-1b66-4af7-a603-41c15bdd804b-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-xpnfj\" (UID: \"9af7eca1-1b66-4af7-a603-41c15bdd804b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" Sep 08 09:11:28.530138 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.529956 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9af7eca1-1b66-4af7-a603-41c15bdd804b-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-xpnfj\" (UID: \"9af7eca1-1b66-4af7-a603-41c15bdd804b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" Sep 08 09:11:28.530138 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.530005 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9af7eca1-1b66-4af7-a603-41c15bdd804b-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-xpnfj\" (UID: \"9af7eca1-1b66-4af7-a603-41c15bdd804b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" Sep 08 09:11:28.530138 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.530025 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4266r\" (UniqueName: \"kubernetes.io/projected/9af7eca1-1b66-4af7-a603-41c15bdd804b-kube-api-access-4266r\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-xpnfj\" (UID: \"9af7eca1-1b66-4af7-a603-41c15bdd804b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" Sep 08 09:11:28.530138 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.530070 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/9af7eca1-1b66-4af7-a603-41c15bdd804b-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-xpnfj\" (UID: \"9af7eca1-1b66-4af7-a603-41c15bdd804b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" Sep 08 09:11:28.630757 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.630723 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/9af7eca1-1b66-4af7-a603-41c15bdd804b-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-xpnfj\" (UID: \"9af7eca1-1b66-4af7-a603-41c15bdd804b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" Sep 08 09:11:28.630948 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.630778 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9af7eca1-1b66-4af7-a603-41c15bdd804b-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-xpnfj\" (UID: \"9af7eca1-1b66-4af7-a603-41c15bdd804b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" Sep 08 09:11:28.630948 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.630800 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9af7eca1-1b66-4af7-a603-41c15bdd804b-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-xpnfj\" (UID: \"9af7eca1-1b66-4af7-a603-41c15bdd804b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" Sep 08 09:11:28.630948 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.630824 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9af7eca1-1b66-4af7-a603-41c15bdd804b-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-xpnfj\" (UID: \"9af7eca1-1b66-4af7-a603-41c15bdd804b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" Sep 08 09:11:28.630948 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.630842 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9af7eca1-1b66-4af7-a603-41c15bdd804b-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-xpnfj\" (UID: \"9af7eca1-1b66-4af7-a603-41c15bdd804b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" Sep 08 09:11:28.630948 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.630883 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4266r\" (UniqueName: \"kubernetes.io/projected/9af7eca1-1b66-4af7-a603-41c15bdd804b-kube-api-access-4266r\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-xpnfj\" (UID: \"9af7eca1-1b66-4af7-a603-41c15bdd804b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" Sep 08 09:11:28.631302 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.631284 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9af7eca1-1b66-4af7-a603-41c15bdd804b-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-xpnfj\" (UID: \"9af7eca1-1b66-4af7-a603-41c15bdd804b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" Sep 08 09:11:28.631533 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.631514 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/9af7eca1-1b66-4af7-a603-41c15bdd804b-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-xpnfj\" (UID: \"9af7eca1-1b66-4af7-a603-41c15bdd804b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" Sep 08 09:11:28.633326 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.633303 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9af7eca1-1b66-4af7-a603-41c15bdd804b-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-xpnfj\" (UID: \"9af7eca1-1b66-4af7-a603-41c15bdd804b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" Sep 08 09:11:28.633429 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.633394 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9af7eca1-1b66-4af7-a603-41c15bdd804b-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-xpnfj\" (UID: \"9af7eca1-1b66-4af7-a603-41c15bdd804b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" Sep 08 09:11:28.633518 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.633425 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9af7eca1-1b66-4af7-a603-41c15bdd804b-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-xpnfj\" (UID: \"9af7eca1-1b66-4af7-a603-41c15bdd804b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" Sep 08 09:11:28.643269 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.643245 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4266r\" (UniqueName: \"kubernetes.io/projected/9af7eca1-1b66-4af7-a603-41c15bdd804b-kube-api-access-4266r\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-xpnfj\" (UID: \"9af7eca1-1b66-4af7-a603-41c15bdd804b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" Sep 08 09:11:28.739369 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:28.739334 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" Sep 08 09:11:29.018236 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:29.018212 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj"] Sep 08 09:11:29.019924 ip-10-0-141-177 kubenswrapper[2554]: W0908 09:11:29.019903 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9af7eca1_1b66_4af7_a603_41c15bdd804b.slice/crio-6c5a2b963046180e62747512ba9a3ba05b59e0903ec6605e57f931c1e1bd1794 WatchSource:0}: Error finding container 6c5a2b963046180e62747512ba9a3ba05b59e0903ec6605e57f931c1e1bd1794: Status 404 returned error can't find the container with id 6c5a2b963046180e62747512ba9a3ba05b59e0903ec6605e57f931c1e1bd1794 Sep 08 09:11:29.230896 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:29.230862 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" event={"ID":"9af7eca1-1b66-4af7-a603-41c15bdd804b","Type":"ContainerStarted","Data":"e15ed18fcf01f633d9dbe45f65feec659e04e01713dbc5422c9a812a9892ee92"} Sep 08 09:11:29.231080 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:29.230907 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" event={"ID":"9af7eca1-1b66-4af7-a603-41c15bdd804b","Type":"ContainerStarted","Data":"6c5a2b963046180e62747512ba9a3ba05b59e0903ec6605e57f931c1e1bd1794"} Sep 08 09:11:34.255789 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:34.255752 2554 generic.go:356] "Generic (PLEG): container finished" podID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerID="e15ed18fcf01f633d9dbe45f65feec659e04e01713dbc5422c9a812a9892ee92" exitCode=0 Sep 08 09:11:34.256170 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:34.255813 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" event={"ID":"9af7eca1-1b66-4af7-a603-41c15bdd804b","Type":"ContainerDied","Data":"e15ed18fcf01f633d9dbe45f65feec659e04e01713dbc5422c9a812a9892ee92"} Sep 08 09:11:35.261844 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:11:35.261805 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" event={"ID":"9af7eca1-1b66-4af7-a603-41c15bdd804b","Type":"ContainerStarted","Data":"622fe72fc36c18342876fe87df443fd9ad023e27adbd86b01a2a40c65a921f7d"} Sep 08 09:12:03.394051 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:12:03.394017 2554 generic.go:356] "Generic (PLEG): container finished" podID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerID="622fe72fc36c18342876fe87df443fd9ad023e27adbd86b01a2a40c65a921f7d" exitCode=0 Sep 08 09:12:03.394513 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:12:03.394087 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" event={"ID":"9af7eca1-1b66-4af7-a603-41c15bdd804b","Type":"ContainerDied","Data":"622fe72fc36c18342876fe87df443fd9ad023e27adbd86b01a2a40c65a921f7d"} Sep 08 09:12:04.402741 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:12:04.402701 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" event={"ID":"9af7eca1-1b66-4af7-a603-41c15bdd804b","Type":"ContainerStarted","Data":"a7699753bc48c5c031a53579dfc83ccf919b60e00bec25c5f7d77cc5ac73b7d2"} Sep 08 09:12:04.403242 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:12:04.402761 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" event={"ID":"9af7eca1-1b66-4af7-a603-41c15bdd804b","Type":"ContainerStarted","Data":"dd0c626a1701be98d8c78a3254a917a34ac0601f7e913a2bc969f4c735a0bc1e"} Sep 08 09:12:04.403242 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:12:04.402782 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" event={"ID":"9af7eca1-1b66-4af7-a603-41c15bdd804b","Type":"ContainerStarted","Data":"092061e9f666ce3f5e3e59475b059d77371e5dec48d21ec8329087a4550ecbd6"} Sep 08 09:12:04.451032 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:12:04.450947 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" podStartSLOduration=36.450925123 podStartE2EDuration="36.450925123s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 09:11:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 09:12:04.450139532 +0000 UTC m=+1470.958246302" watchObservedRunningTime="2026-09-08 09:12:04.450925123 +0000 UTC m=+1470.959031893" Sep 08 09:12:04.740188 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:12:04.740135 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" Sep 08 09:12:04.740391 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:12:04.740201 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" Sep 08 09:12:04.740391 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:12:04.740218 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" Sep 08 09:12:04.741878 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:12:04.741836 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" podUID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.29:8816: connect: connection refused" Sep 08 09:12:04.742031 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:12:04.741893 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" podUID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerName="online" probeResult="failure" output="Get \"https://10.132.0.29:6567/health\": dial tcp 10.132.0.29:6567: connect: connection refused" Sep 08 09:12:04.742031 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:12:04.741903 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" podUID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.29:8443: connect: connection refused" Sep 08 09:12:07.740777 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:12:07.740723 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" podUID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.29:8443: connect: connection refused" Sep 08 09:12:07.741310 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:12:07.740736 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" podUID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerName="online" probeResult="failure" output="Get \"https://10.132.0.29:6567/health\": dial tcp 10.132.0.29:6567: connect: connection refused" Sep 08 09:12:07.741310 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:12:07.740725 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" podUID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.29:8816: connect: connection refused" Sep 08 09:12:08.739723 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:12:08.739676 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" Sep 08 09:12:08.739933 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:12:08.739745 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" Sep 08 09:12:08.739933 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:12:08.739765 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" Sep 08 09:12:10.740463 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:12:10.740417 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" podUID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.29:8443: connect: connection refused" Sep 08 09:12:10.740950 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:12:10.740418 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" podUID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.29:8816: connect: connection refused" Sep 08 09:12:10.740950 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:12:10.740495 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" podUID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerName="online" probeResult="failure" output="Get \"https://10.132.0.29:6567/health\": dial tcp 10.132.0.29:6567: connect: connection refused" Sep 08 09:12:13.741251 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:12:13.741221 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" Sep 08 09:12:13.741740 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:12:13.741319 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" Sep 08 09:12:13.742122 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:12:13.742103 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" Sep 08 09:12:13.742198 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:12:13.742163 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" Sep 08 09:12:14.744171 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:12:14.744123 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" podUID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerName="online" probeResult="failure" output="Get \"https://10.132.0.29:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 08 09:12:17.745174 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:12:17.745123 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" podUID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerName="online" probeResult="failure" output="Get \"https://10.132.0.29:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 08 09:12:20.694964 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:12:20.694922 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" Sep 08 09:12:21.699139 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:12:21.699098 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" podUID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerName="online" probeResult="failure" output="Get \"https://10.132.0.29:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 08 09:12:31.675737 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:12:31.675705 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" Sep 08 09:12:34.012329 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:12:34.012300 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xnd5g_df2da370-7d8d-414d-b82f-3d857fadd070/console-operator/2.log" Sep 08 09:12:34.015612 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:12:34.015578 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xnd5g_df2da370-7d8d-414d-b82f-3d857fadd070/console-operator/2.log" Sep 08 09:12:34.016798 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:12:34.016776 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xn2lh_04c4d313-ed7e-4fea-bec5-3ed11857a8c4/ovn-acl-logging/0.log" Sep 08 09:12:34.020424 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:12:34.020401 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xn2lh_04c4d313-ed7e-4fea-bec5-3ed11857a8c4/ovn-acl-logging/0.log" Sep 08 09:13:32.806418 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:13:32.806382 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj"] Sep 08 09:13:32.806915 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:13:32.806686 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" podUID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerName="online" containerID="cri-o://092061e9f666ce3f5e3e59475b059d77371e5dec48d21ec8329087a4550ecbd6" gracePeriod=30 Sep 08 09:13:32.806915 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:13:32.806778 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" podUID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerName="offline" containerID="cri-o://dd0c626a1701be98d8c78a3254a917a34ac0601f7e913a2bc969f4c735a0bc1e" gracePeriod=30 Sep 08 09:13:32.806915 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:13:32.806778 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" podUID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerName="ui" containerID="cri-o://a7699753bc48c5c031a53579dfc83ccf919b60e00bec25c5f7d77cc5ac73b7d2" gracePeriod=30 Sep 08 09:13:33.026062 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:13:33.026028 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc"] Sep 08 09:13:33.026447 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:13:33.026277 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" podUID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerName="registry" containerID="cri-o://cb1f53fa0e646b669833afdbf93c1d34e2c6c4ba6f1f5d3f8434f499cd6f9c7d" gracePeriod=30 Sep 08 09:13:33.026447 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:13:33.026341 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" podUID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerName="offline" containerID="cri-o://5282045ed519618fa730d2c026e2072ba8d0b70a61529ef83ede0c5eeebeff4d" gracePeriod=30 Sep 08 09:13:33.026447 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:13:33.026381 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" podUID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerName="online" containerID="cri-o://512480e0e02a9d5bbf671938894d3e2a95cbf0be52b6439019ada3ee64c7ea44" gracePeriod=30 Sep 08 09:13:33.026698 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:13:33.026341 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" podUID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerName="ui" containerID="cri-o://0d9b5e32057bf82af5170442cba560367897187748a2172aef5b83bda7966924" gracePeriod=30 Sep 08 09:13:33.742669 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:13:33.742622 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" podUID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.29:8443: connect: connection refused" Sep 08 09:13:33.802782 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:13:33.802752 2554 generic.go:356] "Generic (PLEG): container finished" podID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerID="0d9b5e32057bf82af5170442cba560367897187748a2172aef5b83bda7966924" exitCode=0 Sep 08 09:13:33.802895 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:13:33.802829 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" event={"ID":"9a288e10-6517-4367-8f4d-9621b23a01cf","Type":"ContainerDied","Data":"0d9b5e32057bf82af5170442cba560367897187748a2172aef5b83bda7966924"} Sep 08 09:13:33.805488 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:13:33.805469 2554 generic.go:356] "Generic (PLEG): container finished" podID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerID="a7699753bc48c5c031a53579dfc83ccf919b60e00bec25c5f7d77cc5ac73b7d2" exitCode=0 Sep 08 09:13:33.805488 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:13:33.805486 2554 generic.go:356] "Generic (PLEG): container finished" podID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerID="092061e9f666ce3f5e3e59475b059d77371e5dec48d21ec8329087a4550ecbd6" exitCode=0 Sep 08 09:13:33.805629 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:13:33.805520 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" event={"ID":"9af7eca1-1b66-4af7-a603-41c15bdd804b","Type":"ContainerDied","Data":"a7699753bc48c5c031a53579dfc83ccf919b60e00bec25c5f7d77cc5ac73b7d2"} Sep 08 09:13:33.805629 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:13:33.805545 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" event={"ID":"9af7eca1-1b66-4af7-a603-41c15bdd804b","Type":"ContainerDied","Data":"092061e9f666ce3f5e3e59475b059d77371e5dec48d21ec8329087a4550ecbd6"} Sep 08 09:13:34.761527 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:13:34.761479 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" podUID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.28:8443: connect: connection refused" Sep 08 09:13:34.813664 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:13:34.813631 2554 generic.go:356] "Generic (PLEG): container finished" podID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerID="512480e0e02a9d5bbf671938894d3e2a95cbf0be52b6439019ada3ee64c7ea44" exitCode=0 Sep 08 09:13:34.813797 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:13:34.813696 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" event={"ID":"9a288e10-6517-4367-8f4d-9621b23a01cf","Type":"ContainerDied","Data":"512480e0e02a9d5bbf671938894d3e2a95cbf0be52b6439019ada3ee64c7ea44"} Sep 08 09:13:34.951997 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:13:34.951936 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" podUID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerName="online" probeResult="failure" output="Get \"https://10.132.0.28:6567/health\": dial tcp 10.132.0.28:6567: connect: connection refused" Sep 08 09:13:40.696601 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:13:40.696558 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" podUID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerName="online" probeResult="failure" output="Get \"https://10.132.0.29:6567/health\": dial tcp 10.132.0.29:6567: connect: connection refused" Sep 08 09:13:43.741988 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:13:43.741942 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" podUID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.29:8443: connect: connection refused" Sep 08 09:13:44.760646 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:13:44.760599 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" podUID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.28:8443: connect: connection refused" Sep 08 09:13:44.952054 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:13:44.952011 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" podUID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerName="online" probeResult="failure" output="Get \"https://10.132.0.28:6567/health\": dial tcp 10.132.0.28:6567: connect: connection refused" Sep 08 09:13:50.696203 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:13:50.696152 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" podUID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerName="online" probeResult="failure" output="Get \"https://10.132.0.29:6567/health\": dial tcp 10.132.0.29:6567: connect: connection refused" Sep 08 09:13:53.742146 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:13:53.742105 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" podUID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.29:8443: connect: connection refused" Sep 08 09:13:53.742545 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:13:53.742219 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" Sep 08 09:13:54.761412 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:13:54.761369 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" podUID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.28:8443: connect: connection refused" Sep 08 09:13:54.761829 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:13:54.761519 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:13:54.951521 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:13:54.951477 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" podUID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerName="online" probeResult="failure" output="Get \"https://10.132.0.28:6567/health\": dial tcp 10.132.0.28:6567: connect: connection refused" Sep 08 09:13:54.951713 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:13:54.951615 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:14:00.696439 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:00.696399 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" podUID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerName="online" probeResult="failure" output="Get \"https://10.132.0.29:6567/health\": dial tcp 10.132.0.29:6567: connect: connection refused" Sep 08 09:14:00.696910 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:00.696519 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" Sep 08 09:14:02.938902 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:02.938870 2554 generic.go:356] "Generic (PLEG): container finished" podID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerID="dd0c626a1701be98d8c78a3254a917a34ac0601f7e913a2bc969f4c735a0bc1e" exitCode=137 Sep 08 09:14:02.939264 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:02.938941 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" event={"ID":"9af7eca1-1b66-4af7-a603-41c15bdd804b","Type":"ContainerDied","Data":"dd0c626a1701be98d8c78a3254a917a34ac0601f7e913a2bc969f4c735a0bc1e"} Sep 08 09:14:03.459940 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.459910 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" Sep 08 09:14:03.564688 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.564649 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9af7eca1-1b66-4af7-a603-41c15bdd804b-online-tls\") pod \"9af7eca1-1b66-4af7-a603-41c15bdd804b\" (UID: \"9af7eca1-1b66-4af7-a603-41c15bdd804b\") " Sep 08 09:14:03.564808 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.564732 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4266r\" (UniqueName: \"kubernetes.io/projected/9af7eca1-1b66-4af7-a603-41c15bdd804b-kube-api-access-4266r\") pod \"9af7eca1-1b66-4af7-a603-41c15bdd804b\" (UID: \"9af7eca1-1b66-4af7-a603-41c15bdd804b\") " Sep 08 09:14:03.564808 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.564788 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9af7eca1-1b66-4af7-a603-41c15bdd804b-offline-tls\") pod \"9af7eca1-1b66-4af7-a603-41c15bdd804b\" (UID: \"9af7eca1-1b66-4af7-a603-41c15bdd804b\") " Sep 08 09:14:03.564917 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.564827 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9af7eca1-1b66-4af7-a603-41c15bdd804b-feast-data\") pod \"9af7eca1-1b66-4af7-a603-41c15bdd804b\" (UID: \"9af7eca1-1b66-4af7-a603-41c15bdd804b\") " Sep 08 09:14:03.564917 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.564867 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/9af7eca1-1b66-4af7-a603-41c15bdd804b-registry-tls\") pod \"9af7eca1-1b66-4af7-a603-41c15bdd804b\" (UID: \"9af7eca1-1b66-4af7-a603-41c15bdd804b\") " Sep 08 09:14:03.565054 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.564915 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9af7eca1-1b66-4af7-a603-41c15bdd804b-ui-tls\") pod \"9af7eca1-1b66-4af7-a603-41c15bdd804b\" (UID: \"9af7eca1-1b66-4af7-a603-41c15bdd804b\") " Sep 08 09:14:03.565291 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.565260 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9af7eca1-1b66-4af7-a603-41c15bdd804b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "9af7eca1-1b66-4af7-a603-41c15bdd804b" (UID: "9af7eca1-1b66-4af7-a603-41c15bdd804b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 09:14:03.565390 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.565353 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9af7eca1-1b66-4af7-a603-41c15bdd804b-feast-data" (OuterVolumeSpecName: "feast-data") pod "9af7eca1-1b66-4af7-a603-41c15bdd804b" (UID: "9af7eca1-1b66-4af7-a603-41c15bdd804b"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 09:14:03.567128 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.567100 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9af7eca1-1b66-4af7-a603-41c15bdd804b-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "9af7eca1-1b66-4af7-a603-41c15bdd804b" (UID: "9af7eca1-1b66-4af7-a603-41c15bdd804b"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 09:14:03.567128 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.567114 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9af7eca1-1b66-4af7-a603-41c15bdd804b-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "9af7eca1-1b66-4af7-a603-41c15bdd804b" (UID: "9af7eca1-1b66-4af7-a603-41c15bdd804b"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 09:14:03.567271 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.567193 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9af7eca1-1b66-4af7-a603-41c15bdd804b-online-tls" (OuterVolumeSpecName: "online-tls") pod "9af7eca1-1b66-4af7-a603-41c15bdd804b" (UID: "9af7eca1-1b66-4af7-a603-41c15bdd804b"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 09:14:03.567330 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.567308 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9af7eca1-1b66-4af7-a603-41c15bdd804b-kube-api-access-4266r" (OuterVolumeSpecName: "kube-api-access-4266r") pod "9af7eca1-1b66-4af7-a603-41c15bdd804b" (UID: "9af7eca1-1b66-4af7-a603-41c15bdd804b"). InnerVolumeSpecName "kube-api-access-4266r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 09:14:03.666165 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.666137 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:14:03.666304 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.666221 2554 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9af7eca1-1b66-4af7-a603-41c15bdd804b-offline-tls\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 09:14:03.666304 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.666244 2554 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9af7eca1-1b66-4af7-a603-41c15bdd804b-feast-data\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 09:14:03.666304 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.666256 2554 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/9af7eca1-1b66-4af7-a603-41c15bdd804b-registry-tls\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 09:14:03.666304 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.666268 2554 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9af7eca1-1b66-4af7-a603-41c15bdd804b-ui-tls\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 09:14:03.666304 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.666279 2554 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9af7eca1-1b66-4af7-a603-41c15bdd804b-online-tls\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 09:14:03.666304 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.666290 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4266r\" (UniqueName: \"kubernetes.io/projected/9af7eca1-1b66-4af7-a603-41c15bdd804b-kube-api-access-4266r\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 09:14:03.766716 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.766690 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9a288e10-6517-4367-8f4d-9621b23a01cf-online-tls\") pod \"9a288e10-6517-4367-8f4d-9621b23a01cf\" (UID: \"9a288e10-6517-4367-8f4d-9621b23a01cf\") " Sep 08 09:14:03.766871 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.766750 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9a288e10-6517-4367-8f4d-9621b23a01cf-ui-tls\") pod \"9a288e10-6517-4367-8f4d-9621b23a01cf\" (UID: \"9a288e10-6517-4367-8f4d-9621b23a01cf\") " Sep 08 09:14:03.766871 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.766798 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9a288e10-6517-4367-8f4d-9621b23a01cf-feast-data\") pod \"9a288e10-6517-4367-8f4d-9621b23a01cf\" (UID: \"9a288e10-6517-4367-8f4d-9621b23a01cf\") " Sep 08 09:14:03.766871 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.766820 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9a288e10-6517-4367-8f4d-9621b23a01cf-offline-tls\") pod \"9a288e10-6517-4367-8f4d-9621b23a01cf\" (UID: \"9a288e10-6517-4367-8f4d-9621b23a01cf\") " Sep 08 09:14:03.766871 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.766839 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dn7k7\" (UniqueName: \"kubernetes.io/projected/9a288e10-6517-4367-8f4d-9621b23a01cf-kube-api-access-dn7k7\") pod \"9a288e10-6517-4367-8f4d-9621b23a01cf\" (UID: \"9a288e10-6517-4367-8f4d-9621b23a01cf\") " Sep 08 09:14:03.767129 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.766912 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/9a288e10-6517-4367-8f4d-9621b23a01cf-registry-tls\") pod \"9a288e10-6517-4367-8f4d-9621b23a01cf\" (UID: \"9a288e10-6517-4367-8f4d-9621b23a01cf\") " Sep 08 09:14:03.767404 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.767374 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a288e10-6517-4367-8f4d-9621b23a01cf-feast-data" (OuterVolumeSpecName: "feast-data") pod "9a288e10-6517-4367-8f4d-9621b23a01cf" (UID: "9a288e10-6517-4367-8f4d-9621b23a01cf"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 09:14:03.768997 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.768948 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a288e10-6517-4367-8f4d-9621b23a01cf-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "9a288e10-6517-4367-8f4d-9621b23a01cf" (UID: "9a288e10-6517-4367-8f4d-9621b23a01cf"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 09:14:03.769095 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.769007 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a288e10-6517-4367-8f4d-9621b23a01cf-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "9a288e10-6517-4367-8f4d-9621b23a01cf" (UID: "9a288e10-6517-4367-8f4d-9621b23a01cf"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 09:14:03.769095 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.769017 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a288e10-6517-4367-8f4d-9621b23a01cf-kube-api-access-dn7k7" (OuterVolumeSpecName: "kube-api-access-dn7k7") pod "9a288e10-6517-4367-8f4d-9621b23a01cf" (UID: "9a288e10-6517-4367-8f4d-9621b23a01cf"). InnerVolumeSpecName "kube-api-access-dn7k7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 09:14:03.769256 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.769235 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a288e10-6517-4367-8f4d-9621b23a01cf-online-tls" (OuterVolumeSpecName: "online-tls") pod "9a288e10-6517-4367-8f4d-9621b23a01cf" (UID: "9a288e10-6517-4367-8f4d-9621b23a01cf"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 09:14:03.769309 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.769243 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a288e10-6517-4367-8f4d-9621b23a01cf-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "9a288e10-6517-4367-8f4d-9621b23a01cf" (UID: "9a288e10-6517-4367-8f4d-9621b23a01cf"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 09:14:03.868061 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.867974 2554 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9a288e10-6517-4367-8f4d-9621b23a01cf-online-tls\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 09:14:03.868061 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.868019 2554 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9a288e10-6517-4367-8f4d-9621b23a01cf-ui-tls\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 09:14:03.868061 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.868031 2554 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9a288e10-6517-4367-8f4d-9621b23a01cf-feast-data\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 09:14:03.868061 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.868042 2554 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9a288e10-6517-4367-8f4d-9621b23a01cf-offline-tls\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 09:14:03.868061 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.868055 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dn7k7\" (UniqueName: \"kubernetes.io/projected/9a288e10-6517-4367-8f4d-9621b23a01cf-kube-api-access-dn7k7\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 09:14:03.868286 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.868069 2554 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/9a288e10-6517-4367-8f4d-9621b23a01cf-registry-tls\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 09:14:03.945588 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.945569 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" Sep 08 09:14:03.945588 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.945578 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj" event={"ID":"9af7eca1-1b66-4af7-a603-41c15bdd804b","Type":"ContainerDied","Data":"6c5a2b963046180e62747512ba9a3ba05b59e0903ec6605e57f931c1e1bd1794"} Sep 08 09:14:03.946003 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.945637 2554 scope.go:117] "RemoveContainer" containerID="a7699753bc48c5c031a53579dfc83ccf919b60e00bec25c5f7d77cc5ac73b7d2" Sep 08 09:14:03.948852 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.948825 2554 generic.go:356] "Generic (PLEG): container finished" podID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerID="5282045ed519618fa730d2c026e2072ba8d0b70a61529ef83ede0c5eeebeff4d" exitCode=137 Sep 08 09:14:03.948852 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.948849 2554 generic.go:356] "Generic (PLEG): container finished" podID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerID="cb1f53fa0e646b669833afdbf93c1d34e2c6c4ba6f1f5d3f8434f499cd6f9c7d" exitCode=137 Sep 08 09:14:03.949031 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.948880 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" Sep 08 09:14:03.949031 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.948884 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" event={"ID":"9a288e10-6517-4367-8f4d-9621b23a01cf","Type":"ContainerDied","Data":"5282045ed519618fa730d2c026e2072ba8d0b70a61529ef83ede0c5eeebeff4d"} Sep 08 09:14:03.949031 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.948932 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" event={"ID":"9a288e10-6517-4367-8f4d-9621b23a01cf","Type":"ContainerDied","Data":"cb1f53fa0e646b669833afdbf93c1d34e2c6c4ba6f1f5d3f8434f499cd6f9c7d"} Sep 08 09:14:03.949031 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.948956 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc" event={"ID":"9a288e10-6517-4367-8f4d-9621b23a01cf","Type":"ContainerDied","Data":"1725f6ba60c346971836810801cd68fe3cfd3bb143078d489e100e33cae21b6e"} Sep 08 09:14:03.956165 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.956147 2554 scope.go:117] "RemoveContainer" containerID="dd0c626a1701be98d8c78a3254a917a34ac0601f7e913a2bc969f4c735a0bc1e" Sep 08 09:14:03.964046 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.964026 2554 scope.go:117] "RemoveContainer" containerID="092061e9f666ce3f5e3e59475b059d77371e5dec48d21ec8329087a4550ecbd6" Sep 08 09:14:03.971142 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.971118 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj"] Sep 08 09:14:03.972872 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.972855 2554 scope.go:117] "RemoveContainer" containerID="622fe72fc36c18342876fe87df443fd9ad023e27adbd86b01a2a40c65a921f7d" Sep 08 09:14:03.977926 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.977903 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-xpnfj"] Sep 08 09:14:03.980316 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.980303 2554 scope.go:117] "RemoveContainer" containerID="e15ed18fcf01f633d9dbe45f65feec659e04e01713dbc5422c9a812a9892ee92" Sep 08 09:14:03.987144 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.987124 2554 scope.go:117] "RemoveContainer" containerID="0d9b5e32057bf82af5170442cba560367897187748a2172aef5b83bda7966924" Sep 08 09:14:03.989809 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.989789 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc"] Sep 08 09:14:03.994951 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.994933 2554 scope.go:117] "RemoveContainer" containerID="5282045ed519618fa730d2c026e2072ba8d0b70a61529ef83ede0c5eeebeff4d" Sep 08 09:14:03.995040 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:03.994942 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wzvc"] Sep 08 09:14:04.002089 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:04.002074 2554 scope.go:117] "RemoveContainer" containerID="512480e0e02a9d5bbf671938894d3e2a95cbf0be52b6439019ada3ee64c7ea44" Sep 08 09:14:04.008782 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:04.008764 2554 scope.go:117] "RemoveContainer" containerID="cb1f53fa0e646b669833afdbf93c1d34e2c6c4ba6f1f5d3f8434f499cd6f9c7d" Sep 08 09:14:04.015609 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:04.015592 2554 scope.go:117] "RemoveContainer" containerID="a56ad026927d1cff04f97601439843dc714cc899a85450bd81bff2bb3e69eba8" Sep 08 09:14:04.022356 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:04.022339 2554 scope.go:117] "RemoveContainer" containerID="dffaa2341a0bb67c0e56f1194ae924c06143e83c4fc624dd9404d56a2c8c3449" Sep 08 09:14:04.029637 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:04.029619 2554 scope.go:117] "RemoveContainer" containerID="0d9b5e32057bf82af5170442cba560367897187748a2172aef5b83bda7966924" Sep 08 09:14:04.029899 ip-10-0-141-177 kubenswrapper[2554]: E0908 09:14:04.029882 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d9b5e32057bf82af5170442cba560367897187748a2172aef5b83bda7966924\": container with ID starting with 0d9b5e32057bf82af5170442cba560367897187748a2172aef5b83bda7966924 not found: ID does not exist" containerID="0d9b5e32057bf82af5170442cba560367897187748a2172aef5b83bda7966924" Sep 08 09:14:04.029971 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:04.029904 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d9b5e32057bf82af5170442cba560367897187748a2172aef5b83bda7966924"} err="failed to get container status \"0d9b5e32057bf82af5170442cba560367897187748a2172aef5b83bda7966924\": rpc error: code = NotFound desc = could not find container \"0d9b5e32057bf82af5170442cba560367897187748a2172aef5b83bda7966924\": container with ID starting with 0d9b5e32057bf82af5170442cba560367897187748a2172aef5b83bda7966924 not found: ID does not exist" Sep 08 09:14:04.029971 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:04.029920 2554 scope.go:117] "RemoveContainer" containerID="5282045ed519618fa730d2c026e2072ba8d0b70a61529ef83ede0c5eeebeff4d" Sep 08 09:14:04.030202 ip-10-0-141-177 kubenswrapper[2554]: E0908 09:14:04.030181 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5282045ed519618fa730d2c026e2072ba8d0b70a61529ef83ede0c5eeebeff4d\": container with ID starting with 5282045ed519618fa730d2c026e2072ba8d0b70a61529ef83ede0c5eeebeff4d not found: ID does not exist" containerID="5282045ed519618fa730d2c026e2072ba8d0b70a61529ef83ede0c5eeebeff4d" Sep 08 09:14:04.030296 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:04.030208 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5282045ed519618fa730d2c026e2072ba8d0b70a61529ef83ede0c5eeebeff4d"} err="failed to get container status \"5282045ed519618fa730d2c026e2072ba8d0b70a61529ef83ede0c5eeebeff4d\": rpc error: code = NotFound desc = could not find container \"5282045ed519618fa730d2c026e2072ba8d0b70a61529ef83ede0c5eeebeff4d\": container with ID starting with 5282045ed519618fa730d2c026e2072ba8d0b70a61529ef83ede0c5eeebeff4d not found: ID does not exist" Sep 08 09:14:04.030296 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:04.030229 2554 scope.go:117] "RemoveContainer" containerID="512480e0e02a9d5bbf671938894d3e2a95cbf0be52b6439019ada3ee64c7ea44" Sep 08 09:14:04.030480 ip-10-0-141-177 kubenswrapper[2554]: E0908 09:14:04.030459 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"512480e0e02a9d5bbf671938894d3e2a95cbf0be52b6439019ada3ee64c7ea44\": container with ID starting with 512480e0e02a9d5bbf671938894d3e2a95cbf0be52b6439019ada3ee64c7ea44 not found: ID does not exist" containerID="512480e0e02a9d5bbf671938894d3e2a95cbf0be52b6439019ada3ee64c7ea44" Sep 08 09:14:04.030530 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:04.030488 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"512480e0e02a9d5bbf671938894d3e2a95cbf0be52b6439019ada3ee64c7ea44"} err="failed to get container status \"512480e0e02a9d5bbf671938894d3e2a95cbf0be52b6439019ada3ee64c7ea44\": rpc error: code = NotFound desc = could not find container \"512480e0e02a9d5bbf671938894d3e2a95cbf0be52b6439019ada3ee64c7ea44\": container with ID starting with 512480e0e02a9d5bbf671938894d3e2a95cbf0be52b6439019ada3ee64c7ea44 not found: ID does not exist" Sep 08 09:14:04.030530 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:04.030509 2554 scope.go:117] "RemoveContainer" containerID="cb1f53fa0e646b669833afdbf93c1d34e2c6c4ba6f1f5d3f8434f499cd6f9c7d" Sep 08 09:14:04.030733 ip-10-0-141-177 kubenswrapper[2554]: E0908 09:14:04.030717 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb1f53fa0e646b669833afdbf93c1d34e2c6c4ba6f1f5d3f8434f499cd6f9c7d\": container with ID starting with cb1f53fa0e646b669833afdbf93c1d34e2c6c4ba6f1f5d3f8434f499cd6f9c7d not found: ID does not exist" containerID="cb1f53fa0e646b669833afdbf93c1d34e2c6c4ba6f1f5d3f8434f499cd6f9c7d" Sep 08 09:14:04.030801 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:04.030741 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb1f53fa0e646b669833afdbf93c1d34e2c6c4ba6f1f5d3f8434f499cd6f9c7d"} err="failed to get container status \"cb1f53fa0e646b669833afdbf93c1d34e2c6c4ba6f1f5d3f8434f499cd6f9c7d\": rpc error: code = NotFound desc = could not find container \"cb1f53fa0e646b669833afdbf93c1d34e2c6c4ba6f1f5d3f8434f499cd6f9c7d\": container with ID starting with cb1f53fa0e646b669833afdbf93c1d34e2c6c4ba6f1f5d3f8434f499cd6f9c7d not found: ID does not exist" Sep 08 09:14:04.030801 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:04.030760 2554 scope.go:117] "RemoveContainer" containerID="a56ad026927d1cff04f97601439843dc714cc899a85450bd81bff2bb3e69eba8" Sep 08 09:14:04.031037 ip-10-0-141-177 kubenswrapper[2554]: E0908 09:14:04.031015 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a56ad026927d1cff04f97601439843dc714cc899a85450bd81bff2bb3e69eba8\": container with ID starting with a56ad026927d1cff04f97601439843dc714cc899a85450bd81bff2bb3e69eba8 not found: ID does not exist" containerID="a56ad026927d1cff04f97601439843dc714cc899a85450bd81bff2bb3e69eba8" Sep 08 09:14:04.031138 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:04.031043 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a56ad026927d1cff04f97601439843dc714cc899a85450bd81bff2bb3e69eba8"} err="failed to get container status \"a56ad026927d1cff04f97601439843dc714cc899a85450bd81bff2bb3e69eba8\": rpc error: code = NotFound desc = could not find container \"a56ad026927d1cff04f97601439843dc714cc899a85450bd81bff2bb3e69eba8\": container with ID starting with a56ad026927d1cff04f97601439843dc714cc899a85450bd81bff2bb3e69eba8 not found: ID does not exist" Sep 08 09:14:04.031138 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:04.031062 2554 scope.go:117] "RemoveContainer" containerID="dffaa2341a0bb67c0e56f1194ae924c06143e83c4fc624dd9404d56a2c8c3449" Sep 08 09:14:04.031309 ip-10-0-141-177 kubenswrapper[2554]: E0908 09:14:04.031292 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dffaa2341a0bb67c0e56f1194ae924c06143e83c4fc624dd9404d56a2c8c3449\": container with ID starting with dffaa2341a0bb67c0e56f1194ae924c06143e83c4fc624dd9404d56a2c8c3449 not found: ID does not exist" containerID="dffaa2341a0bb67c0e56f1194ae924c06143e83c4fc624dd9404d56a2c8c3449" Sep 08 09:14:04.031358 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:04.031316 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dffaa2341a0bb67c0e56f1194ae924c06143e83c4fc624dd9404d56a2c8c3449"} err="failed to get container status \"dffaa2341a0bb67c0e56f1194ae924c06143e83c4fc624dd9404d56a2c8c3449\": rpc error: code = NotFound desc = could not find container \"dffaa2341a0bb67c0e56f1194ae924c06143e83c4fc624dd9404d56a2c8c3449\": container with ID starting with dffaa2341a0bb67c0e56f1194ae924c06143e83c4fc624dd9404d56a2c8c3449 not found: ID does not exist" Sep 08 09:14:04.031358 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:04.031337 2554 scope.go:117] "RemoveContainer" containerID="0d9b5e32057bf82af5170442cba560367897187748a2172aef5b83bda7966924" Sep 08 09:14:04.031576 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:04.031558 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d9b5e32057bf82af5170442cba560367897187748a2172aef5b83bda7966924"} err="failed to get container status \"0d9b5e32057bf82af5170442cba560367897187748a2172aef5b83bda7966924\": rpc error: code = NotFound desc = could not find container \"0d9b5e32057bf82af5170442cba560367897187748a2172aef5b83bda7966924\": container with ID starting with 0d9b5e32057bf82af5170442cba560367897187748a2172aef5b83bda7966924 not found: ID does not exist" Sep 08 09:14:04.031646 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:04.031578 2554 scope.go:117] "RemoveContainer" containerID="5282045ed519618fa730d2c026e2072ba8d0b70a61529ef83ede0c5eeebeff4d" Sep 08 09:14:04.031834 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:04.031816 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5282045ed519618fa730d2c026e2072ba8d0b70a61529ef83ede0c5eeebeff4d"} err="failed to get container status \"5282045ed519618fa730d2c026e2072ba8d0b70a61529ef83ede0c5eeebeff4d\": rpc error: code = NotFound desc = could not find container \"5282045ed519618fa730d2c026e2072ba8d0b70a61529ef83ede0c5eeebeff4d\": container with ID starting with 5282045ed519618fa730d2c026e2072ba8d0b70a61529ef83ede0c5eeebeff4d not found: ID does not exist" Sep 08 09:14:04.031884 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:04.031836 2554 scope.go:117] "RemoveContainer" containerID="512480e0e02a9d5bbf671938894d3e2a95cbf0be52b6439019ada3ee64c7ea44" Sep 08 09:14:04.032110 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:04.032085 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"512480e0e02a9d5bbf671938894d3e2a95cbf0be52b6439019ada3ee64c7ea44"} err="failed to get container status \"512480e0e02a9d5bbf671938894d3e2a95cbf0be52b6439019ada3ee64c7ea44\": rpc error: code = NotFound desc = could not find container \"512480e0e02a9d5bbf671938894d3e2a95cbf0be52b6439019ada3ee64c7ea44\": container with ID starting with 512480e0e02a9d5bbf671938894d3e2a95cbf0be52b6439019ada3ee64c7ea44 not found: ID does not exist" Sep 08 09:14:04.032171 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:04.032112 2554 scope.go:117] "RemoveContainer" containerID="cb1f53fa0e646b669833afdbf93c1d34e2c6c4ba6f1f5d3f8434f499cd6f9c7d" Sep 08 09:14:04.032339 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:04.032322 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb1f53fa0e646b669833afdbf93c1d34e2c6c4ba6f1f5d3f8434f499cd6f9c7d"} err="failed to get container status \"cb1f53fa0e646b669833afdbf93c1d34e2c6c4ba6f1f5d3f8434f499cd6f9c7d\": rpc error: code = NotFound desc = could not find container \"cb1f53fa0e646b669833afdbf93c1d34e2c6c4ba6f1f5d3f8434f499cd6f9c7d\": container with ID starting with cb1f53fa0e646b669833afdbf93c1d34e2c6c4ba6f1f5d3f8434f499cd6f9c7d not found: ID does not exist" Sep 08 09:14:04.032403 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:04.032340 2554 scope.go:117] "RemoveContainer" containerID="a56ad026927d1cff04f97601439843dc714cc899a85450bd81bff2bb3e69eba8" Sep 08 09:14:04.032572 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:04.032547 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a56ad026927d1cff04f97601439843dc714cc899a85450bd81bff2bb3e69eba8"} err="failed to get container status \"a56ad026927d1cff04f97601439843dc714cc899a85450bd81bff2bb3e69eba8\": rpc error: code = NotFound desc = could not find container \"a56ad026927d1cff04f97601439843dc714cc899a85450bd81bff2bb3e69eba8\": container with ID starting with a56ad026927d1cff04f97601439843dc714cc899a85450bd81bff2bb3e69eba8 not found: ID does not exist" Sep 08 09:14:04.032619 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:04.032575 2554 scope.go:117] "RemoveContainer" containerID="dffaa2341a0bb67c0e56f1194ae924c06143e83c4fc624dd9404d56a2c8c3449" Sep 08 09:14:04.032792 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:04.032775 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dffaa2341a0bb67c0e56f1194ae924c06143e83c4fc624dd9404d56a2c8c3449"} err="failed to get container status \"dffaa2341a0bb67c0e56f1194ae924c06143e83c4fc624dd9404d56a2c8c3449\": rpc error: code = NotFound desc = could not find container \"dffaa2341a0bb67c0e56f1194ae924c06143e83c4fc624dd9404d56a2c8c3449\": container with ID starting with dffaa2341a0bb67c0e56f1194ae924c06143e83c4fc624dd9404d56a2c8c3449 not found: ID does not exist" Sep 08 09:14:04.034045 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:04.034028 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a288e10-6517-4367-8f4d-9621b23a01cf" path="/var/lib/kubelet/pods/9a288e10-6517-4367-8f4d-9621b23a01cf/volumes" Sep 08 09:14:04.034618 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:04.034606 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9af7eca1-1b66-4af7-a603-41c15bdd804b" path="/var/lib/kubelet/pods/9af7eca1-1b66-4af7-a603-41c15bdd804b/volumes" Sep 08 09:14:44.431375 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.431344 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb"] Sep 08 09:14:44.431798 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.431657 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerName="feast-apply" Sep 08 09:14:44.431798 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.431669 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerName="feast-apply" Sep 08 09:14:44.431798 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.431676 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerName="feast-init" Sep 08 09:14:44.431798 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.431682 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerName="feast-init" Sep 08 09:14:44.431798 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.431690 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerName="feast-apply" Sep 08 09:14:44.431798 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.431697 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerName="feast-apply" Sep 08 09:14:44.431798 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.431710 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerName="online" Sep 08 09:14:44.431798 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.431715 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerName="online" Sep 08 09:14:44.431798 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.431725 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerName="ui" Sep 08 09:14:44.431798 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.431731 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerName="ui" Sep 08 09:14:44.431798 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.431738 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerName="registry" Sep 08 09:14:44.431798 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.431743 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerName="registry" Sep 08 09:14:44.431798 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.431750 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerName="online" Sep 08 09:14:44.431798 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.431755 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerName="online" Sep 08 09:14:44.431798 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.431765 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerName="ui" Sep 08 09:14:44.431798 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.431770 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerName="ui" Sep 08 09:14:44.431798 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.431777 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerName="offline" Sep 08 09:14:44.431798 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.431782 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerName="offline" Sep 08 09:14:44.431798 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.431788 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerName="feast-init" Sep 08 09:14:44.431798 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.431793 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerName="feast-init" Sep 08 09:14:44.431798 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.431800 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerName="offline" Sep 08 09:14:44.431798 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.431805 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerName="offline" Sep 08 09:14:44.432614 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.431851 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerName="offline" Sep 08 09:14:44.432614 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.431859 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerName="offline" Sep 08 09:14:44.432614 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.431865 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerName="registry" Sep 08 09:14:44.432614 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.431870 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerName="online" Sep 08 09:14:44.432614 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.431876 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerName="online" Sep 08 09:14:44.432614 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.431881 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a288e10-6517-4367-8f4d-9621b23a01cf" containerName="ui" Sep 08 09:14:44.432614 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.431888 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="9af7eca1-1b66-4af7-a603-41c15bdd804b" containerName="ui" Sep 08 09:14:44.436737 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.436721 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:14:44.439432 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.439409 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-wn8t9\"" Sep 08 09:14:44.439572 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.439518 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 08 09:14:44.439697 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.439673 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 08 09:14:44.439815 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.439694 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 08 09:14:44.439815 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.439801 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 08 09:14:44.439912 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.439848 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 08 09:14:44.440037 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.440022 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 08 09:14:44.453565 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.453545 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb"] Sep 08 09:14:44.593897 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.593867 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/595f9cfb-4eaf-415a-96ae-c531642b3179-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9j8qb\" (UID: \"595f9cfb-4eaf-415a-96ae-c531642b3179\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:14:44.593897 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.593899 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/595f9cfb-4eaf-415a-96ae-c531642b3179-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9j8qb\" (UID: \"595f9cfb-4eaf-415a-96ae-c531642b3179\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:14:44.594102 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.593925 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/595f9cfb-4eaf-415a-96ae-c531642b3179-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9j8qb\" (UID: \"595f9cfb-4eaf-415a-96ae-c531642b3179\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:14:44.594102 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.594030 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/595f9cfb-4eaf-415a-96ae-c531642b3179-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-9j8qb\" (UID: \"595f9cfb-4eaf-415a-96ae-c531642b3179\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:14:44.594102 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.594064 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhrtn\" (UniqueName: \"kubernetes.io/projected/595f9cfb-4eaf-415a-96ae-c531642b3179-kube-api-access-mhrtn\") pod \"feast-simple-feast-setup-57576b8fcf-9j8qb\" (UID: \"595f9cfb-4eaf-415a-96ae-c531642b3179\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:14:44.594211 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.594102 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/595f9cfb-4eaf-415a-96ae-c531642b3179-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9j8qb\" (UID: \"595f9cfb-4eaf-415a-96ae-c531642b3179\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:14:44.694729 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.694660 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/595f9cfb-4eaf-415a-96ae-c531642b3179-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9j8qb\" (UID: \"595f9cfb-4eaf-415a-96ae-c531642b3179\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:14:44.694729 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.694699 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/595f9cfb-4eaf-415a-96ae-c531642b3179-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9j8qb\" (UID: \"595f9cfb-4eaf-415a-96ae-c531642b3179\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:14:44.694880 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.694738 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/595f9cfb-4eaf-415a-96ae-c531642b3179-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-9j8qb\" (UID: \"595f9cfb-4eaf-415a-96ae-c531642b3179\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:14:44.694880 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.694773 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhrtn\" (UniqueName: \"kubernetes.io/projected/595f9cfb-4eaf-415a-96ae-c531642b3179-kube-api-access-mhrtn\") pod \"feast-simple-feast-setup-57576b8fcf-9j8qb\" (UID: \"595f9cfb-4eaf-415a-96ae-c531642b3179\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:14:44.694880 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.694819 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/595f9cfb-4eaf-415a-96ae-c531642b3179-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9j8qb\" (UID: \"595f9cfb-4eaf-415a-96ae-c531642b3179\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:14:44.694880 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.694860 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/595f9cfb-4eaf-415a-96ae-c531642b3179-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9j8qb\" (UID: \"595f9cfb-4eaf-415a-96ae-c531642b3179\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:14:44.695168 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.695147 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/595f9cfb-4eaf-415a-96ae-c531642b3179-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-9j8qb\" (UID: \"595f9cfb-4eaf-415a-96ae-c531642b3179\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:14:44.697381 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.697359 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/595f9cfb-4eaf-415a-96ae-c531642b3179-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9j8qb\" (UID: \"595f9cfb-4eaf-415a-96ae-c531642b3179\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:14:44.697504 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.697385 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/595f9cfb-4eaf-415a-96ae-c531642b3179-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9j8qb\" (UID: \"595f9cfb-4eaf-415a-96ae-c531642b3179\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:14:44.697504 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.697387 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/595f9cfb-4eaf-415a-96ae-c531642b3179-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9j8qb\" (UID: \"595f9cfb-4eaf-415a-96ae-c531642b3179\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:14:44.697504 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.697387 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/595f9cfb-4eaf-415a-96ae-c531642b3179-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9j8qb\" (UID: \"595f9cfb-4eaf-415a-96ae-c531642b3179\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:14:44.703391 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.703369 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhrtn\" (UniqueName: \"kubernetes.io/projected/595f9cfb-4eaf-415a-96ae-c531642b3179-kube-api-access-mhrtn\") pod \"feast-simple-feast-setup-57576b8fcf-9j8qb\" (UID: \"595f9cfb-4eaf-415a-96ae-c531642b3179\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:14:44.760430 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:44.760405 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:14:45.037074 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:45.037049 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb"] Sep 08 09:14:45.039213 ip-10-0-141-177 kubenswrapper[2554]: W0908 09:14:45.039187 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod595f9cfb_4eaf_415a_96ae_c531642b3179.slice/crio-4797b414a356a0f6059219d3aac0391ba63a99ad3021552816b5482c7fbdf01b WatchSource:0}: Error finding container 4797b414a356a0f6059219d3aac0391ba63a99ad3021552816b5482c7fbdf01b: Status 404 returned error can't find the container with id 4797b414a356a0f6059219d3aac0391ba63a99ad3021552816b5482c7fbdf01b Sep 08 09:14:45.121643 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:45.121611 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" event={"ID":"595f9cfb-4eaf-415a-96ae-c531642b3179","Type":"ContainerStarted","Data":"d9897655705ede9e16b128f3cea9bdd7b212b4eca49de24baed39cf5368a4d18"} Sep 08 09:14:45.121758 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:45.121666 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" event={"ID":"595f9cfb-4eaf-415a-96ae-c531642b3179","Type":"ContainerStarted","Data":"4797b414a356a0f6059219d3aac0391ba63a99ad3021552816b5482c7fbdf01b"} Sep 08 09:14:50.144295 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:50.144200 2554 generic.go:356] "Generic (PLEG): container finished" podID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerID="d9897655705ede9e16b128f3cea9bdd7b212b4eca49de24baed39cf5368a4d18" exitCode=0 Sep 08 09:14:50.144295 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:50.144248 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" event={"ID":"595f9cfb-4eaf-415a-96ae-c531642b3179","Type":"ContainerDied","Data":"d9897655705ede9e16b128f3cea9bdd7b212b4eca49de24baed39cf5368a4d18"} Sep 08 09:14:51.150229 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:51.150195 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" event={"ID":"595f9cfb-4eaf-415a-96ae-c531642b3179","Type":"ContainerStarted","Data":"239fd8ae39ee90534b6da8312514a9850eea99e19e0ab878e84e5b48e48f803b"} Sep 08 09:14:56.173947 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:56.173867 2554 generic.go:356] "Generic (PLEG): container finished" podID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerID="239fd8ae39ee90534b6da8312514a9850eea99e19e0ab878e84e5b48e48f803b" exitCode=0 Sep 08 09:14:56.174307 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:56.173939 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" event={"ID":"595f9cfb-4eaf-415a-96ae-c531642b3179","Type":"ContainerDied","Data":"239fd8ae39ee90534b6da8312514a9850eea99e19e0ab878e84e5b48e48f803b"} Sep 08 09:14:57.186109 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:57.186064 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" event={"ID":"595f9cfb-4eaf-415a-96ae-c531642b3179","Type":"ContainerStarted","Data":"f97cfa2375709da875c0d2d1218803ae8f81ed29136873ad4f85177a6717ad53"} Sep 08 09:14:57.186652 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:57.186137 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" event={"ID":"595f9cfb-4eaf-415a-96ae-c531642b3179","Type":"ContainerStarted","Data":"580e0100f8420159034b80445314223b7f1794f02d761cd900f8c7d7b3cb8705"} Sep 08 09:14:57.186652 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:57.186160 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" event={"ID":"595f9cfb-4eaf-415a-96ae-c531642b3179","Type":"ContainerStarted","Data":"418c55dfddd6705301dacbad8a181cd2c51092e9f0ee5c07b500e31e1e900b13"} Sep 08 09:14:57.186652 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:57.186181 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" event={"ID":"595f9cfb-4eaf-415a-96ae-c531642b3179","Type":"ContainerStarted","Data":"c933312a5b690a2ffa399f94edadf5297571f2de0ba38e1a3d1bfdb87bb4d84b"} Sep 08 09:14:57.212998 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:57.212914 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" podStartSLOduration=13.212893873 podStartE2EDuration="13.212893873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 09:14:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 09:14:57.210124878 +0000 UTC m=+1643.718231646" watchObservedRunningTime="2026-09-08 09:14:57.212893873 +0000 UTC m=+1643.721000641" Sep 08 09:14:59.761050 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:59.761010 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:14:59.761604 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:59.761584 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:14:59.761757 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:59.761746 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:14:59.762051 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:59.762023 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:14:59.763291 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:59.763252 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" podUID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.30:6571: connect: connection refused" Sep 08 09:14:59.763291 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:59.763269 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" podUID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.30:8816: connect: connection refused" Sep 08 09:14:59.763472 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:59.763346 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" podUID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.30:8443: connect: connection refused" Sep 08 09:14:59.763472 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:14:59.763386 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" podUID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": dial tcp 10.132.0.30:6567: connect: connection refused" Sep 08 09:15:02.761663 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:15:02.761614 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" podUID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.30:6571: connect: connection refused" Sep 08 09:15:02.762766 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:15:02.761630 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" podUID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.30:8443: connect: connection refused" Sep 08 09:15:02.763051 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:15:02.761638 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" podUID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": dial tcp 10.132.0.30:6567: connect: connection refused" Sep 08 09:15:02.763209 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:15:02.762125 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" podUID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.30:8816: connect: connection refused" Sep 08 09:15:04.761278 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:15:04.761233 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:15:04.761278 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:15:04.761289 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:15:04.761700 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:15:04.761303 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:15:04.761700 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:15:04.761316 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:15:05.762223 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:15:05.762185 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:15:05.762817 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:15:05.762793 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:15:05.762973 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:15:05.762952 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:15:05.763110 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:15:05.763029 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:15:05.767893 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:15:05.767867 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:15:06.241028 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:15:06.241001 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:15:06.241211 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:15:06.241093 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:15:06.243587 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:15:06.243571 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:16:07.772389 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:07.772287 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb"] Sep 08 09:16:07.772974 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:07.772545 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" podUID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerName="registry" containerID="cri-o://c933312a5b690a2ffa399f94edadf5297571f2de0ba38e1a3d1bfdb87bb4d84b" gracePeriod=30 Sep 08 09:16:07.772974 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:07.772657 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" podUID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerName="ui" containerID="cri-o://f97cfa2375709da875c0d2d1218803ae8f81ed29136873ad4f85177a6717ad53" gracePeriod=30 Sep 08 09:16:07.772974 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:07.772680 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" podUID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerName="offline" containerID="cri-o://580e0100f8420159034b80445314223b7f1794f02d761cd900f8c7d7b3cb8705" gracePeriod=30 Sep 08 09:16:07.772974 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:07.772666 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" podUID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerName="online" containerID="cri-o://418c55dfddd6705301dacbad8a181cd2c51092e9f0ee5c07b500e31e1e900b13" gracePeriod=30 Sep 08 09:16:08.230217 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:08.230174 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn"] Sep 08 09:16:08.234951 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:08.234919 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:16:08.237422 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:08.237393 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-vwwbg\"" Sep 08 09:16:08.245739 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:08.245711 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn"] Sep 08 09:16:08.311713 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:08.311666 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/52bcc27d-3095-424f-99f6-64f8d7dae87b-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fxqbn\" (UID: \"52bcc27d-3095-424f-99f6-64f8d7dae87b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:16:08.311713 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:08.311708 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddcsq\" (UniqueName: \"kubernetes.io/projected/52bcc27d-3095-424f-99f6-64f8d7dae87b-kube-api-access-ddcsq\") pod \"feast-simple-feast-setup-57576b8fcf-fxqbn\" (UID: \"52bcc27d-3095-424f-99f6-64f8d7dae87b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:16:08.311997 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:08.311792 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/52bcc27d-3095-424f-99f6-64f8d7dae87b-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fxqbn\" (UID: \"52bcc27d-3095-424f-99f6-64f8d7dae87b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:16:08.311997 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:08.311814 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/52bcc27d-3095-424f-99f6-64f8d7dae87b-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fxqbn\" (UID: \"52bcc27d-3095-424f-99f6-64f8d7dae87b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:16:08.311997 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:08.311836 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/52bcc27d-3095-424f-99f6-64f8d7dae87b-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-fxqbn\" (UID: \"52bcc27d-3095-424f-99f6-64f8d7dae87b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:16:08.311997 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:08.311874 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/52bcc27d-3095-424f-99f6-64f8d7dae87b-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fxqbn\" (UID: \"52bcc27d-3095-424f-99f6-64f8d7dae87b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:16:08.412864 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:08.412817 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/52bcc27d-3095-424f-99f6-64f8d7dae87b-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fxqbn\" (UID: \"52bcc27d-3095-424f-99f6-64f8d7dae87b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:16:08.412864 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:08.412856 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddcsq\" (UniqueName: \"kubernetes.io/projected/52bcc27d-3095-424f-99f6-64f8d7dae87b-kube-api-access-ddcsq\") pod \"feast-simple-feast-setup-57576b8fcf-fxqbn\" (UID: \"52bcc27d-3095-424f-99f6-64f8d7dae87b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:16:08.413119 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:08.412905 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/52bcc27d-3095-424f-99f6-64f8d7dae87b-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fxqbn\" (UID: \"52bcc27d-3095-424f-99f6-64f8d7dae87b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:16:08.413119 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:08.412924 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/52bcc27d-3095-424f-99f6-64f8d7dae87b-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fxqbn\" (UID: \"52bcc27d-3095-424f-99f6-64f8d7dae87b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:16:08.413119 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:08.412941 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/52bcc27d-3095-424f-99f6-64f8d7dae87b-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-fxqbn\" (UID: \"52bcc27d-3095-424f-99f6-64f8d7dae87b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:16:08.413119 ip-10-0-141-177 kubenswrapper[2554]: E0908 09:16:08.412971 2554 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Sep 08 09:16:08.413119 ip-10-0-141-177 kubenswrapper[2554]: E0908 09:16:08.413072 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52bcc27d-3095-424f-99f6-64f8d7dae87b-ui-tls podName:52bcc27d-3095-424f-99f6-64f8d7dae87b nodeName:}" failed. No retries permitted until 2026-09-08 09:16:08.913048318 +0000 UTC m=+1715.421155077 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/52bcc27d-3095-424f-99f6-64f8d7dae87b-ui-tls") pod "feast-simple-feast-setup-57576b8fcf-fxqbn" (UID: "52bcc27d-3095-424f-99f6-64f8d7dae87b") : secret "feast-simple-feast-setup-ui-tls" not found Sep 08 09:16:08.413119 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:08.412975 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/52bcc27d-3095-424f-99f6-64f8d7dae87b-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fxqbn\" (UID: \"52bcc27d-3095-424f-99f6-64f8d7dae87b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:16:08.413483 ip-10-0-141-177 kubenswrapper[2554]: E0908 09:16:08.413152 2554 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found Sep 08 09:16:08.413483 ip-10-0-141-177 kubenswrapper[2554]: E0908 09:16:08.413223 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52bcc27d-3095-424f-99f6-64f8d7dae87b-online-tls podName:52bcc27d-3095-424f-99f6-64f8d7dae87b nodeName:}" failed. No retries permitted until 2026-09-08 09:16:08.913204677 +0000 UTC m=+1715.421311425 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/52bcc27d-3095-424f-99f6-64f8d7dae87b-online-tls") pod "feast-simple-feast-setup-57576b8fcf-fxqbn" (UID: "52bcc27d-3095-424f-99f6-64f8d7dae87b") : secret "feast-simple-feast-setup-online-tls" not found Sep 08 09:16:08.413483 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:08.413339 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/52bcc27d-3095-424f-99f6-64f8d7dae87b-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-fxqbn\" (UID: \"52bcc27d-3095-424f-99f6-64f8d7dae87b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:16:08.415448 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:08.415424 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/52bcc27d-3095-424f-99f6-64f8d7dae87b-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fxqbn\" (UID: \"52bcc27d-3095-424f-99f6-64f8d7dae87b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:16:08.415602 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:08.415577 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/52bcc27d-3095-424f-99f6-64f8d7dae87b-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fxqbn\" (UID: \"52bcc27d-3095-424f-99f6-64f8d7dae87b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:16:08.421825 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:08.421798 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddcsq\" (UniqueName: \"kubernetes.io/projected/52bcc27d-3095-424f-99f6-64f8d7dae87b-kube-api-access-ddcsq\") pod \"feast-simple-feast-setup-57576b8fcf-fxqbn\" (UID: \"52bcc27d-3095-424f-99f6-64f8d7dae87b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:16:08.509702 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:08.509666 2554 generic.go:356] "Generic (PLEG): container finished" podID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerID="f97cfa2375709da875c0d2d1218803ae8f81ed29136873ad4f85177a6717ad53" exitCode=0 Sep 08 09:16:08.509912 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:08.509739 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" event={"ID":"595f9cfb-4eaf-415a-96ae-c531642b3179","Type":"ContainerDied","Data":"f97cfa2375709da875c0d2d1218803ae8f81ed29136873ad4f85177a6717ad53"} Sep 08 09:16:08.917085 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:08.917003 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/52bcc27d-3095-424f-99f6-64f8d7dae87b-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fxqbn\" (UID: \"52bcc27d-3095-424f-99f6-64f8d7dae87b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:16:08.917085 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:08.917044 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/52bcc27d-3095-424f-99f6-64f8d7dae87b-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fxqbn\" (UID: \"52bcc27d-3095-424f-99f6-64f8d7dae87b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:16:08.919468 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:08.919441 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/52bcc27d-3095-424f-99f6-64f8d7dae87b-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fxqbn\" (UID: \"52bcc27d-3095-424f-99f6-64f8d7dae87b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:16:08.919592 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:08.919441 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/52bcc27d-3095-424f-99f6-64f8d7dae87b-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fxqbn\" (UID: \"52bcc27d-3095-424f-99f6-64f8d7dae87b\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:16:09.144962 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:09.144926 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:16:09.424076 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:09.424049 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn"] Sep 08 09:16:09.425813 ip-10-0-141-177 kubenswrapper[2554]: W0908 09:16:09.425784 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod52bcc27d_3095_424f_99f6_64f8d7dae87b.slice/crio-9d98c277e17d462603c3b2fe13a4f5bff1c343d7c9969d34e9036b9dcfd05821 WatchSource:0}: Error finding container 9d98c277e17d462603c3b2fe13a4f5bff1c343d7c9969d34e9036b9dcfd05821: Status 404 returned error can't find the container with id 9d98c277e17d462603c3b2fe13a4f5bff1c343d7c9969d34e9036b9dcfd05821 Sep 08 09:16:09.517841 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:09.517811 2554 generic.go:356] "Generic (PLEG): container finished" podID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerID="418c55dfddd6705301dacbad8a181cd2c51092e9f0ee5c07b500e31e1e900b13" exitCode=0 Sep 08 09:16:09.517948 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:09.517879 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" event={"ID":"595f9cfb-4eaf-415a-96ae-c531642b3179","Type":"ContainerDied","Data":"418c55dfddd6705301dacbad8a181cd2c51092e9f0ee5c07b500e31e1e900b13"} Sep 08 09:16:09.519420 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:09.519399 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" event={"ID":"52bcc27d-3095-424f-99f6-64f8d7dae87b","Type":"ContainerStarted","Data":"0f3f340eac5eeaed2d403b5dd41276179f0d2b027fb9835b2b23e1e921740a21"} Sep 08 09:16:09.519534 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:09.519430 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" event={"ID":"52bcc27d-3095-424f-99f6-64f8d7dae87b","Type":"ContainerStarted","Data":"9d98c277e17d462603c3b2fe13a4f5bff1c343d7c9969d34e9036b9dcfd05821"} Sep 08 09:16:14.545768 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:14.545679 2554 generic.go:356] "Generic (PLEG): container finished" podID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerID="0f3f340eac5eeaed2d403b5dd41276179f0d2b027fb9835b2b23e1e921740a21" exitCode=0 Sep 08 09:16:14.545768 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:14.545720 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" event={"ID":"52bcc27d-3095-424f-99f6-64f8d7dae87b","Type":"ContainerDied","Data":"0f3f340eac5eeaed2d403b5dd41276179f0d2b027fb9835b2b23e1e921740a21"} Sep 08 09:16:15.551833 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:15.551788 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" event={"ID":"52bcc27d-3095-424f-99f6-64f8d7dae87b","Type":"ContainerStarted","Data":"27e1957886956331bead3d8e2c6513b03c02b3889bef217d92ba5b8862154b00"} Sep 08 09:16:16.240622 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:16.240583 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" podUID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.30:8443: connect: connection refused" Sep 08 09:16:16.240821 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:16.240656 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" podUID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": dial tcp 10.132.0.30:6567: connect: connection refused" Sep 08 09:16:20.577421 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:20.577381 2554 generic.go:356] "Generic (PLEG): container finished" podID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerID="27e1957886956331bead3d8e2c6513b03c02b3889bef217d92ba5b8862154b00" exitCode=0 Sep 08 09:16:20.577792 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:20.577452 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" event={"ID":"52bcc27d-3095-424f-99f6-64f8d7dae87b","Type":"ContainerDied","Data":"27e1957886956331bead3d8e2c6513b03c02b3889bef217d92ba5b8862154b00"} Sep 08 09:16:21.589223 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:21.589167 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" event={"ID":"52bcc27d-3095-424f-99f6-64f8d7dae87b","Type":"ContainerStarted","Data":"e4b76f6e0d2cdbb0f96db13738ea2bd3b7168117c0b572db7c9e8673cfe3d075"} Sep 08 09:16:21.589734 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:21.589275 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" event={"ID":"52bcc27d-3095-424f-99f6-64f8d7dae87b","Type":"ContainerStarted","Data":"534ea69c54e683f1a817d9ddc5d8ee83cb370cbf3211f4223ba39837352832ec"} Sep 08 09:16:21.589734 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:21.589302 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" event={"ID":"52bcc27d-3095-424f-99f6-64f8d7dae87b","Type":"ContainerStarted","Data":"44f74f38fc80a298e67899dd4a5fa4c39fd2e33c947c4354f205d1462d4c8f41"} Sep 08 09:16:21.589734 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:21.589319 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" event={"ID":"52bcc27d-3095-424f-99f6-64f8d7dae87b","Type":"ContainerStarted","Data":"574d43311c11830e649ede911310a7194d20b920ddbc853dfca2e38bd640d075"} Sep 08 09:16:21.616090 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:21.616028 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" podStartSLOduration=13.616004899 podStartE2EDuration="13.616004899s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 09:16:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 09:16:21.615005389 +0000 UTC m=+1728.123112149" watchObservedRunningTime="2026-09-08 09:16:21.616004899 +0000 UTC m=+1728.124111664" Sep 08 09:16:24.145904 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:24.145857 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:16:24.146415 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:24.145934 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:16:24.146415 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:24.145955 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:16:24.146415 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:24.145973 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:16:24.148274 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:24.148225 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" podUID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.31:8816: connect: connection refused" Sep 08 09:16:24.148429 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:24.148263 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" podUID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.31:8443: connect: connection refused" Sep 08 09:16:24.148429 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:24.148258 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" podUID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.31:6571: connect: connection refused" Sep 08 09:16:24.148429 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:24.148326 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" podUID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerName="online" probeResult="failure" output="Get \"https://10.132.0.31:6567/health\": dial tcp 10.132.0.31:6567: connect: connection refused" Sep 08 09:16:26.241420 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:26.241376 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" podUID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.30:8443: connect: connection refused" Sep 08 09:16:26.242019 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:26.241449 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" podUID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": dial tcp 10.132.0.30:6567: connect: connection refused" Sep 08 09:16:27.146096 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:27.146054 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" podUID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.31:8816: connect: connection refused" Sep 08 09:16:27.146435 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:27.146054 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" podUID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.31:6571: connect: connection refused" Sep 08 09:16:27.146435 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:27.146054 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" podUID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.31:8443: connect: connection refused" Sep 08 09:16:27.146653 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:27.146094 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" podUID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerName="online" probeResult="failure" output="Get \"https://10.132.0.31:6567/health\": dial tcp 10.132.0.31:6567: connect: connection refused" Sep 08 09:16:29.145724 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:29.145688 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:16:29.146138 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:29.145738 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:16:29.146138 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:29.145751 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:16:29.146138 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:29.145762 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:16:30.146087 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:30.146055 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:16:30.146486 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:30.146174 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:16:30.146486 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:30.146386 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:16:30.147045 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:30.146997 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:16:30.147154 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:30.147063 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:16:30.147154 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:30.147110 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:16:30.151307 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:30.151289 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:16:30.646510 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:30.646480 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:16:36.241488 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:36.241441 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" podUID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.30:8443: connect: connection refused" Sep 08 09:16:36.241488 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:36.241476 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" podUID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": dial tcp 10.132.0.30:6567: connect: connection refused" Sep 08 09:16:36.242060 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:36.241578 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:16:36.242060 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:36.241652 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:16:38.418169 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.418125 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:16:38.496681 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.496646 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/595f9cfb-4eaf-415a-96ae-c531642b3179-offline-tls\") pod \"595f9cfb-4eaf-415a-96ae-c531642b3179\" (UID: \"595f9cfb-4eaf-415a-96ae-c531642b3179\") " Sep 08 09:16:38.496865 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.496696 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/595f9cfb-4eaf-415a-96ae-c531642b3179-feast-data\") pod \"595f9cfb-4eaf-415a-96ae-c531642b3179\" (UID: \"595f9cfb-4eaf-415a-96ae-c531642b3179\") " Sep 08 09:16:38.496865 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.496736 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/595f9cfb-4eaf-415a-96ae-c531642b3179-ui-tls\") pod \"595f9cfb-4eaf-415a-96ae-c531642b3179\" (UID: \"595f9cfb-4eaf-415a-96ae-c531642b3179\") " Sep 08 09:16:38.496865 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.496760 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mhrtn\" (UniqueName: \"kubernetes.io/projected/595f9cfb-4eaf-415a-96ae-c531642b3179-kube-api-access-mhrtn\") pod \"595f9cfb-4eaf-415a-96ae-c531642b3179\" (UID: \"595f9cfb-4eaf-415a-96ae-c531642b3179\") " Sep 08 09:16:38.496865 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.496797 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/595f9cfb-4eaf-415a-96ae-c531642b3179-registry-tls\") pod \"595f9cfb-4eaf-415a-96ae-c531642b3179\" (UID: \"595f9cfb-4eaf-415a-96ae-c531642b3179\") " Sep 08 09:16:38.497402 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.497355 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/595f9cfb-4eaf-415a-96ae-c531642b3179-feast-data" (OuterVolumeSpecName: "feast-data") pod "595f9cfb-4eaf-415a-96ae-c531642b3179" (UID: "595f9cfb-4eaf-415a-96ae-c531642b3179"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 09:16:38.499094 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.499059 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/595f9cfb-4eaf-415a-96ae-c531642b3179-kube-api-access-mhrtn" (OuterVolumeSpecName: "kube-api-access-mhrtn") pod "595f9cfb-4eaf-415a-96ae-c531642b3179" (UID: "595f9cfb-4eaf-415a-96ae-c531642b3179"). InnerVolumeSpecName "kube-api-access-mhrtn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 09:16:38.499094 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.499088 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/595f9cfb-4eaf-415a-96ae-c531642b3179-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "595f9cfb-4eaf-415a-96ae-c531642b3179" (UID: "595f9cfb-4eaf-415a-96ae-c531642b3179"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 09:16:38.499252 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.499105 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/595f9cfb-4eaf-415a-96ae-c531642b3179-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "595f9cfb-4eaf-415a-96ae-c531642b3179" (UID: "595f9cfb-4eaf-415a-96ae-c531642b3179"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 09:16:38.499252 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.499193 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/595f9cfb-4eaf-415a-96ae-c531642b3179-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "595f9cfb-4eaf-415a-96ae-c531642b3179" (UID: "595f9cfb-4eaf-415a-96ae-c531642b3179"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 09:16:38.597418 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.597330 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/595f9cfb-4eaf-415a-96ae-c531642b3179-online-tls\") pod \"595f9cfb-4eaf-415a-96ae-c531642b3179\" (UID: \"595f9cfb-4eaf-415a-96ae-c531642b3179\") " Sep 08 09:16:38.597633 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.597616 2554 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/595f9cfb-4eaf-415a-96ae-c531642b3179-ui-tls\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 09:16:38.597715 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.597642 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhrtn\" (UniqueName: \"kubernetes.io/projected/595f9cfb-4eaf-415a-96ae-c531642b3179-kube-api-access-mhrtn\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 09:16:38.597715 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.597656 2554 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/595f9cfb-4eaf-415a-96ae-c531642b3179-registry-tls\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 09:16:38.597715 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.597671 2554 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/595f9cfb-4eaf-415a-96ae-c531642b3179-offline-tls\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 09:16:38.597715 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.597683 2554 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/595f9cfb-4eaf-415a-96ae-c531642b3179-feast-data\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 09:16:38.599295 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.599271 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/595f9cfb-4eaf-415a-96ae-c531642b3179-online-tls" (OuterVolumeSpecName: "online-tls") pod "595f9cfb-4eaf-415a-96ae-c531642b3179" (UID: "595f9cfb-4eaf-415a-96ae-c531642b3179"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 09:16:38.679721 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.679693 2554 generic.go:356] "Generic (PLEG): container finished" podID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerID="580e0100f8420159034b80445314223b7f1794f02d761cd900f8c7d7b3cb8705" exitCode=137 Sep 08 09:16:38.679721 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.679716 2554 generic.go:356] "Generic (PLEG): container finished" podID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerID="c933312a5b690a2ffa399f94edadf5297571f2de0ba38e1a3d1bfdb87bb4d84b" exitCode=137 Sep 08 09:16:38.679938 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.679769 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" Sep 08 09:16:38.679938 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.679783 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" event={"ID":"595f9cfb-4eaf-415a-96ae-c531642b3179","Type":"ContainerDied","Data":"580e0100f8420159034b80445314223b7f1794f02d761cd900f8c7d7b3cb8705"} Sep 08 09:16:38.679938 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.679841 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" event={"ID":"595f9cfb-4eaf-415a-96ae-c531642b3179","Type":"ContainerDied","Data":"c933312a5b690a2ffa399f94edadf5297571f2de0ba38e1a3d1bfdb87bb4d84b"} Sep 08 09:16:38.679938 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.679858 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb" event={"ID":"595f9cfb-4eaf-415a-96ae-c531642b3179","Type":"ContainerDied","Data":"4797b414a356a0f6059219d3aac0391ba63a99ad3021552816b5482c7fbdf01b"} Sep 08 09:16:38.679938 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.679879 2554 scope.go:117] "RemoveContainer" containerID="f97cfa2375709da875c0d2d1218803ae8f81ed29136873ad4f85177a6717ad53" Sep 08 09:16:38.688809 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.688793 2554 scope.go:117] "RemoveContainer" containerID="580e0100f8420159034b80445314223b7f1794f02d761cd900f8c7d7b3cb8705" Sep 08 09:16:38.696510 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.696493 2554 scope.go:117] "RemoveContainer" containerID="418c55dfddd6705301dacbad8a181cd2c51092e9f0ee5c07b500e31e1e900b13" Sep 08 09:16:38.698664 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.698639 2554 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/595f9cfb-4eaf-415a-96ae-c531642b3179-online-tls\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 09:16:38.702177 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.702155 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb"] Sep 08 09:16:38.704754 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.704566 2554 scope.go:117] "RemoveContainer" containerID="c933312a5b690a2ffa399f94edadf5297571f2de0ba38e1a3d1bfdb87bb4d84b" Sep 08 09:16:38.706512 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.706491 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j8qb"] Sep 08 09:16:38.712263 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.712230 2554 scope.go:117] "RemoveContainer" containerID="239fd8ae39ee90534b6da8312514a9850eea99e19e0ab878e84e5b48e48f803b" Sep 08 09:16:38.719355 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.719338 2554 scope.go:117] "RemoveContainer" containerID="d9897655705ede9e16b128f3cea9bdd7b212b4eca49de24baed39cf5368a4d18" Sep 08 09:16:38.726373 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.726357 2554 scope.go:117] "RemoveContainer" containerID="f97cfa2375709da875c0d2d1218803ae8f81ed29136873ad4f85177a6717ad53" Sep 08 09:16:38.726637 ip-10-0-141-177 kubenswrapper[2554]: E0908 09:16:38.726617 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f97cfa2375709da875c0d2d1218803ae8f81ed29136873ad4f85177a6717ad53\": container with ID starting with f97cfa2375709da875c0d2d1218803ae8f81ed29136873ad4f85177a6717ad53 not found: ID does not exist" containerID="f97cfa2375709da875c0d2d1218803ae8f81ed29136873ad4f85177a6717ad53" Sep 08 09:16:38.726698 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.726646 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f97cfa2375709da875c0d2d1218803ae8f81ed29136873ad4f85177a6717ad53"} err="failed to get container status \"f97cfa2375709da875c0d2d1218803ae8f81ed29136873ad4f85177a6717ad53\": rpc error: code = NotFound desc = could not find container \"f97cfa2375709da875c0d2d1218803ae8f81ed29136873ad4f85177a6717ad53\": container with ID starting with f97cfa2375709da875c0d2d1218803ae8f81ed29136873ad4f85177a6717ad53 not found: ID does not exist" Sep 08 09:16:38.726698 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.726663 2554 scope.go:117] "RemoveContainer" containerID="580e0100f8420159034b80445314223b7f1794f02d761cd900f8c7d7b3cb8705" Sep 08 09:16:38.726910 ip-10-0-141-177 kubenswrapper[2554]: E0908 09:16:38.726892 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"580e0100f8420159034b80445314223b7f1794f02d761cd900f8c7d7b3cb8705\": container with ID starting with 580e0100f8420159034b80445314223b7f1794f02d761cd900f8c7d7b3cb8705 not found: ID does not exist" containerID="580e0100f8420159034b80445314223b7f1794f02d761cd900f8c7d7b3cb8705" Sep 08 09:16:38.726954 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.726918 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"580e0100f8420159034b80445314223b7f1794f02d761cd900f8c7d7b3cb8705"} err="failed to get container status \"580e0100f8420159034b80445314223b7f1794f02d761cd900f8c7d7b3cb8705\": rpc error: code = NotFound desc = could not find container \"580e0100f8420159034b80445314223b7f1794f02d761cd900f8c7d7b3cb8705\": container with ID starting with 580e0100f8420159034b80445314223b7f1794f02d761cd900f8c7d7b3cb8705 not found: ID does not exist" Sep 08 09:16:38.726954 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.726934 2554 scope.go:117] "RemoveContainer" containerID="418c55dfddd6705301dacbad8a181cd2c51092e9f0ee5c07b500e31e1e900b13" Sep 08 09:16:38.727211 ip-10-0-141-177 kubenswrapper[2554]: E0908 09:16:38.727192 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"418c55dfddd6705301dacbad8a181cd2c51092e9f0ee5c07b500e31e1e900b13\": container with ID starting with 418c55dfddd6705301dacbad8a181cd2c51092e9f0ee5c07b500e31e1e900b13 not found: ID does not exist" containerID="418c55dfddd6705301dacbad8a181cd2c51092e9f0ee5c07b500e31e1e900b13" Sep 08 09:16:38.727260 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.727216 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"418c55dfddd6705301dacbad8a181cd2c51092e9f0ee5c07b500e31e1e900b13"} err="failed to get container status \"418c55dfddd6705301dacbad8a181cd2c51092e9f0ee5c07b500e31e1e900b13\": rpc error: code = NotFound desc = could not find container \"418c55dfddd6705301dacbad8a181cd2c51092e9f0ee5c07b500e31e1e900b13\": container with ID starting with 418c55dfddd6705301dacbad8a181cd2c51092e9f0ee5c07b500e31e1e900b13 not found: ID does not exist" Sep 08 09:16:38.727260 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.727230 2554 scope.go:117] "RemoveContainer" containerID="c933312a5b690a2ffa399f94edadf5297571f2de0ba38e1a3d1bfdb87bb4d84b" Sep 08 09:16:38.727501 ip-10-0-141-177 kubenswrapper[2554]: E0908 09:16:38.727484 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c933312a5b690a2ffa399f94edadf5297571f2de0ba38e1a3d1bfdb87bb4d84b\": container with ID starting with c933312a5b690a2ffa399f94edadf5297571f2de0ba38e1a3d1bfdb87bb4d84b not found: ID does not exist" containerID="c933312a5b690a2ffa399f94edadf5297571f2de0ba38e1a3d1bfdb87bb4d84b" Sep 08 09:16:38.727553 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.727512 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c933312a5b690a2ffa399f94edadf5297571f2de0ba38e1a3d1bfdb87bb4d84b"} err="failed to get container status \"c933312a5b690a2ffa399f94edadf5297571f2de0ba38e1a3d1bfdb87bb4d84b\": rpc error: code = NotFound desc = could not find container \"c933312a5b690a2ffa399f94edadf5297571f2de0ba38e1a3d1bfdb87bb4d84b\": container with ID starting with c933312a5b690a2ffa399f94edadf5297571f2de0ba38e1a3d1bfdb87bb4d84b not found: ID does not exist" Sep 08 09:16:38.727553 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.727526 2554 scope.go:117] "RemoveContainer" containerID="239fd8ae39ee90534b6da8312514a9850eea99e19e0ab878e84e5b48e48f803b" Sep 08 09:16:38.727757 ip-10-0-141-177 kubenswrapper[2554]: E0908 09:16:38.727742 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"239fd8ae39ee90534b6da8312514a9850eea99e19e0ab878e84e5b48e48f803b\": container with ID starting with 239fd8ae39ee90534b6da8312514a9850eea99e19e0ab878e84e5b48e48f803b not found: ID does not exist" containerID="239fd8ae39ee90534b6da8312514a9850eea99e19e0ab878e84e5b48e48f803b" Sep 08 09:16:38.727796 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.727761 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"239fd8ae39ee90534b6da8312514a9850eea99e19e0ab878e84e5b48e48f803b"} err="failed to get container status \"239fd8ae39ee90534b6da8312514a9850eea99e19e0ab878e84e5b48e48f803b\": rpc error: code = NotFound desc = could not find container \"239fd8ae39ee90534b6da8312514a9850eea99e19e0ab878e84e5b48e48f803b\": container with ID starting with 239fd8ae39ee90534b6da8312514a9850eea99e19e0ab878e84e5b48e48f803b not found: ID does not exist" Sep 08 09:16:38.727796 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.727774 2554 scope.go:117] "RemoveContainer" containerID="d9897655705ede9e16b128f3cea9bdd7b212b4eca49de24baed39cf5368a4d18" Sep 08 09:16:38.727973 ip-10-0-141-177 kubenswrapper[2554]: E0908 09:16:38.727959 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9897655705ede9e16b128f3cea9bdd7b212b4eca49de24baed39cf5368a4d18\": container with ID starting with d9897655705ede9e16b128f3cea9bdd7b212b4eca49de24baed39cf5368a4d18 not found: ID does not exist" containerID="d9897655705ede9e16b128f3cea9bdd7b212b4eca49de24baed39cf5368a4d18" Sep 08 09:16:38.728035 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.727989 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9897655705ede9e16b128f3cea9bdd7b212b4eca49de24baed39cf5368a4d18"} err="failed to get container status \"d9897655705ede9e16b128f3cea9bdd7b212b4eca49de24baed39cf5368a4d18\": rpc error: code = NotFound desc = could not find container \"d9897655705ede9e16b128f3cea9bdd7b212b4eca49de24baed39cf5368a4d18\": container with ID starting with d9897655705ede9e16b128f3cea9bdd7b212b4eca49de24baed39cf5368a4d18 not found: ID does not exist" Sep 08 09:16:38.728035 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.728002 2554 scope.go:117] "RemoveContainer" containerID="f97cfa2375709da875c0d2d1218803ae8f81ed29136873ad4f85177a6717ad53" Sep 08 09:16:38.728203 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.728188 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f97cfa2375709da875c0d2d1218803ae8f81ed29136873ad4f85177a6717ad53"} err="failed to get container status \"f97cfa2375709da875c0d2d1218803ae8f81ed29136873ad4f85177a6717ad53\": rpc error: code = NotFound desc = could not find container \"f97cfa2375709da875c0d2d1218803ae8f81ed29136873ad4f85177a6717ad53\": container with ID starting with f97cfa2375709da875c0d2d1218803ae8f81ed29136873ad4f85177a6717ad53 not found: ID does not exist" Sep 08 09:16:38.728266 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.728204 2554 scope.go:117] "RemoveContainer" containerID="580e0100f8420159034b80445314223b7f1794f02d761cd900f8c7d7b3cb8705" Sep 08 09:16:38.728445 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.728430 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"580e0100f8420159034b80445314223b7f1794f02d761cd900f8c7d7b3cb8705"} err="failed to get container status \"580e0100f8420159034b80445314223b7f1794f02d761cd900f8c7d7b3cb8705\": rpc error: code = NotFound desc = could not find container \"580e0100f8420159034b80445314223b7f1794f02d761cd900f8c7d7b3cb8705\": container with ID starting with 580e0100f8420159034b80445314223b7f1794f02d761cd900f8c7d7b3cb8705 not found: ID does not exist" Sep 08 09:16:38.728445 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.728443 2554 scope.go:117] "RemoveContainer" containerID="418c55dfddd6705301dacbad8a181cd2c51092e9f0ee5c07b500e31e1e900b13" Sep 08 09:16:38.728658 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.728641 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"418c55dfddd6705301dacbad8a181cd2c51092e9f0ee5c07b500e31e1e900b13"} err="failed to get container status \"418c55dfddd6705301dacbad8a181cd2c51092e9f0ee5c07b500e31e1e900b13\": rpc error: code = NotFound desc = could not find container \"418c55dfddd6705301dacbad8a181cd2c51092e9f0ee5c07b500e31e1e900b13\": container with ID starting with 418c55dfddd6705301dacbad8a181cd2c51092e9f0ee5c07b500e31e1e900b13 not found: ID does not exist" Sep 08 09:16:38.728658 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.728657 2554 scope.go:117] "RemoveContainer" containerID="c933312a5b690a2ffa399f94edadf5297571f2de0ba38e1a3d1bfdb87bb4d84b" Sep 08 09:16:38.728860 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.728845 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c933312a5b690a2ffa399f94edadf5297571f2de0ba38e1a3d1bfdb87bb4d84b"} err="failed to get container status \"c933312a5b690a2ffa399f94edadf5297571f2de0ba38e1a3d1bfdb87bb4d84b\": rpc error: code = NotFound desc = could not find container \"c933312a5b690a2ffa399f94edadf5297571f2de0ba38e1a3d1bfdb87bb4d84b\": container with ID starting with c933312a5b690a2ffa399f94edadf5297571f2de0ba38e1a3d1bfdb87bb4d84b not found: ID does not exist" Sep 08 09:16:38.728911 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.728860 2554 scope.go:117] "RemoveContainer" containerID="239fd8ae39ee90534b6da8312514a9850eea99e19e0ab878e84e5b48e48f803b" Sep 08 09:16:38.729086 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.729070 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"239fd8ae39ee90534b6da8312514a9850eea99e19e0ab878e84e5b48e48f803b"} err="failed to get container status \"239fd8ae39ee90534b6da8312514a9850eea99e19e0ab878e84e5b48e48f803b\": rpc error: code = NotFound desc = could not find container \"239fd8ae39ee90534b6da8312514a9850eea99e19e0ab878e84e5b48e48f803b\": container with ID starting with 239fd8ae39ee90534b6da8312514a9850eea99e19e0ab878e84e5b48e48f803b not found: ID does not exist" Sep 08 09:16:38.729086 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.729085 2554 scope.go:117] "RemoveContainer" containerID="d9897655705ede9e16b128f3cea9bdd7b212b4eca49de24baed39cf5368a4d18" Sep 08 09:16:38.729264 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:38.729249 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9897655705ede9e16b128f3cea9bdd7b212b4eca49de24baed39cf5368a4d18"} err="failed to get container status \"d9897655705ede9e16b128f3cea9bdd7b212b4eca49de24baed39cf5368a4d18\": rpc error: code = NotFound desc = could not find container \"d9897655705ede9e16b128f3cea9bdd7b212b4eca49de24baed39cf5368a4d18\": container with ID starting with d9897655705ede9e16b128f3cea9bdd7b212b4eca49de24baed39cf5368a4d18 not found: ID does not exist" Sep 08 09:16:40.031702 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:16:40.031668 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="595f9cfb-4eaf-415a-96ae-c531642b3179" path="/var/lib/kubelet/pods/595f9cfb-4eaf-415a-96ae-c531642b3179/volumes" Sep 08 09:17:32.780438 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.780404 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt"] Sep 08 09:17:32.780854 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.780751 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerName="feast-init" Sep 08 09:17:32.780854 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.780764 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerName="feast-init" Sep 08 09:17:32.780854 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.780775 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerName="registry" Sep 08 09:17:32.780854 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.780780 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerName="registry" Sep 08 09:17:32.780854 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.780791 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerName="ui" Sep 08 09:17:32.780854 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.780796 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerName="ui" Sep 08 09:17:32.780854 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.780804 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerName="online" Sep 08 09:17:32.780854 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.780809 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerName="online" Sep 08 09:17:32.780854 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.780821 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerName="feast-apply" Sep 08 09:17:32.780854 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.780827 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerName="feast-apply" Sep 08 09:17:32.780854 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.780843 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerName="offline" Sep 08 09:17:32.780854 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.780851 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerName="offline" Sep 08 09:17:32.781355 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.780925 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerName="online" Sep 08 09:17:32.781355 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.780936 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerName="registry" Sep 08 09:17:32.781355 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.780943 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerName="ui" Sep 08 09:17:32.781355 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.780952 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="595f9cfb-4eaf-415a-96ae-c531642b3179" containerName="offline" Sep 08 09:17:32.784438 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.784419 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" Sep 08 09:17:32.786653 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.786627 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 08 09:17:32.786785 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.786689 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 08 09:17:32.786785 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.786765 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 08 09:17:32.786917 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.786859 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 08 09:17:32.786917 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.786885 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 08 09:17:32.787135 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.787118 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-mrs6p\"" Sep 08 09:17:32.787457 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.787443 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 08 09:17:32.796483 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.796460 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt"] Sep 08 09:17:32.863787 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.863754 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wh9k\" (UniqueName: \"kubernetes.io/projected/3606fbcb-b7cb-4c38-b18a-581f6bec472f-kube-api-access-9wh9k\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-t2fgt\" (UID: \"3606fbcb-b7cb-4c38-b18a-581f6bec472f\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" Sep 08 09:17:32.863943 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.863801 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3606fbcb-b7cb-4c38-b18a-581f6bec472f-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-t2fgt\" (UID: \"3606fbcb-b7cb-4c38-b18a-581f6bec472f\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" Sep 08 09:17:32.863943 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.863831 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3606fbcb-b7cb-4c38-b18a-581f6bec472f-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-t2fgt\" (UID: \"3606fbcb-b7cb-4c38-b18a-581f6bec472f\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" Sep 08 09:17:32.863943 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.863891 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/3606fbcb-b7cb-4c38-b18a-581f6bec472f-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-t2fgt\" (UID: \"3606fbcb-b7cb-4c38-b18a-581f6bec472f\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" Sep 08 09:17:32.863943 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.863923 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3606fbcb-b7cb-4c38-b18a-581f6bec472f-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-t2fgt\" (UID: \"3606fbcb-b7cb-4c38-b18a-581f6bec472f\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" Sep 08 09:17:32.864112 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.863945 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3606fbcb-b7cb-4c38-b18a-581f6bec472f-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-t2fgt\" (UID: \"3606fbcb-b7cb-4c38-b18a-581f6bec472f\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" Sep 08 09:17:32.964481 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.964448 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3606fbcb-b7cb-4c38-b18a-581f6bec472f-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-t2fgt\" (UID: \"3606fbcb-b7cb-4c38-b18a-581f6bec472f\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" Sep 08 09:17:32.964481 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.964486 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3606fbcb-b7cb-4c38-b18a-581f6bec472f-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-t2fgt\" (UID: \"3606fbcb-b7cb-4c38-b18a-581f6bec472f\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" Sep 08 09:17:32.964744 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.964516 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/3606fbcb-b7cb-4c38-b18a-581f6bec472f-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-t2fgt\" (UID: \"3606fbcb-b7cb-4c38-b18a-581f6bec472f\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" Sep 08 09:17:32.964744 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.964533 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3606fbcb-b7cb-4c38-b18a-581f6bec472f-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-t2fgt\" (UID: \"3606fbcb-b7cb-4c38-b18a-581f6bec472f\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" Sep 08 09:17:32.964744 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.964554 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3606fbcb-b7cb-4c38-b18a-581f6bec472f-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-t2fgt\" (UID: \"3606fbcb-b7cb-4c38-b18a-581f6bec472f\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" Sep 08 09:17:32.964744 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.964597 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wh9k\" (UniqueName: \"kubernetes.io/projected/3606fbcb-b7cb-4c38-b18a-581f6bec472f-kube-api-access-9wh9k\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-t2fgt\" (UID: \"3606fbcb-b7cb-4c38-b18a-581f6bec472f\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" Sep 08 09:17:32.964744 ip-10-0-141-177 kubenswrapper[2554]: E0908 09:17:32.964607 2554 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-online-tls: secret "feast-simple-feast-remote-setup-online-tls" not found Sep 08 09:17:32.964744 ip-10-0-141-177 kubenswrapper[2554]: E0908 09:17:32.964686 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3606fbcb-b7cb-4c38-b18a-581f6bec472f-online-tls podName:3606fbcb-b7cb-4c38-b18a-581f6bec472f nodeName:}" failed. No retries permitted until 2026-09-08 09:17:33.46466478 +0000 UTC m=+1799.972771526 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/3606fbcb-b7cb-4c38-b18a-581f6bec472f-online-tls") pod "feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" (UID: "3606fbcb-b7cb-4c38-b18a-581f6bec472f") : secret "feast-simple-feast-remote-setup-online-tls" not found Sep 08 09:17:32.965146 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.965055 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3606fbcb-b7cb-4c38-b18a-581f6bec472f-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-t2fgt\" (UID: \"3606fbcb-b7cb-4c38-b18a-581f6bec472f\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" Sep 08 09:17:32.965272 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.965252 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/3606fbcb-b7cb-4c38-b18a-581f6bec472f-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-t2fgt\" (UID: \"3606fbcb-b7cb-4c38-b18a-581f6bec472f\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" Sep 08 09:17:32.967044 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.967021 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3606fbcb-b7cb-4c38-b18a-581f6bec472f-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-t2fgt\" (UID: \"3606fbcb-b7cb-4c38-b18a-581f6bec472f\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" Sep 08 09:17:32.967150 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.967068 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3606fbcb-b7cb-4c38-b18a-581f6bec472f-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-t2fgt\" (UID: \"3606fbcb-b7cb-4c38-b18a-581f6bec472f\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" Sep 08 09:17:32.973302 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:32.973284 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wh9k\" (UniqueName: \"kubernetes.io/projected/3606fbcb-b7cb-4c38-b18a-581f6bec472f-kube-api-access-9wh9k\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-t2fgt\" (UID: \"3606fbcb-b7cb-4c38-b18a-581f6bec472f\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" Sep 08 09:17:33.469059 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:33.469022 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3606fbcb-b7cb-4c38-b18a-581f6bec472f-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-t2fgt\" (UID: \"3606fbcb-b7cb-4c38-b18a-581f6bec472f\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" Sep 08 09:17:33.471491 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:33.471456 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3606fbcb-b7cb-4c38-b18a-581f6bec472f-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-t2fgt\" (UID: \"3606fbcb-b7cb-4c38-b18a-581f6bec472f\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" Sep 08 09:17:33.693551 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:33.693510 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" Sep 08 09:17:33.969227 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:33.969169 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt"] Sep 08 09:17:33.971472 ip-10-0-141-177 kubenswrapper[2554]: W0908 09:17:33.971423 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3606fbcb_b7cb_4c38_b18a_581f6bec472f.slice/crio-32d0ff8a8e0b606e7f389b411e0e53d36425210b22f83d40d3864e4fa4fdf88e WatchSource:0}: Error finding container 32d0ff8a8e0b606e7f389b411e0e53d36425210b22f83d40d3864e4fa4fdf88e: Status 404 returned error can't find the container with id 32d0ff8a8e0b606e7f389b411e0e53d36425210b22f83d40d3864e4fa4fdf88e Sep 08 09:17:34.048026 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:34.048001 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xnd5g_df2da370-7d8d-414d-b82f-3d857fadd070/console-operator/2.log" Sep 08 09:17:34.053582 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:34.053563 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xnd5g_df2da370-7d8d-414d-b82f-3d857fadd070/console-operator/2.log" Sep 08 09:17:34.053908 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:34.053887 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xn2lh_04c4d313-ed7e-4fea-bec5-3ed11857a8c4/ovn-acl-logging/0.log" Sep 08 09:17:34.058777 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:34.058753 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xn2lh_04c4d313-ed7e-4fea-bec5-3ed11857a8c4/ovn-acl-logging/0.log" Sep 08 09:17:34.923893 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:34.923850 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" event={"ID":"3606fbcb-b7cb-4c38-b18a-581f6bec472f","Type":"ContainerStarted","Data":"6f656d86a9944b6a7bad2a9470e61a6c58a10739e22b0cdee2dab23bf0758fea"} Sep 08 09:17:34.923893 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:34.923899 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" event={"ID":"3606fbcb-b7cb-4c38-b18a-581f6bec472f","Type":"ContainerStarted","Data":"32d0ff8a8e0b606e7f389b411e0e53d36425210b22f83d40d3864e4fa4fdf88e"} Sep 08 09:17:38.943222 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:38.943188 2554 generic.go:356] "Generic (PLEG): container finished" podID="3606fbcb-b7cb-4c38-b18a-581f6bec472f" containerID="6f656d86a9944b6a7bad2a9470e61a6c58a10739e22b0cdee2dab23bf0758fea" exitCode=0 Sep 08 09:17:38.943564 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:38.943236 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" event={"ID":"3606fbcb-b7cb-4c38-b18a-581f6bec472f","Type":"ContainerDied","Data":"6f656d86a9944b6a7bad2a9470e61a6c58a10739e22b0cdee2dab23bf0758fea"} Sep 08 09:17:39.950051 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:17:39.950015 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" event={"ID":"3606fbcb-b7cb-4c38-b18a-581f6bec472f","Type":"ContainerStarted","Data":"3f3acdb2552cc4d32018b9ae293526cd137ac1cb27215e25e261c82f116bdbf3"} Sep 08 09:18:03.056803 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:18:03.056767 2554 generic.go:356] "Generic (PLEG): container finished" podID="3606fbcb-b7cb-4c38-b18a-581f6bec472f" containerID="3f3acdb2552cc4d32018b9ae293526cd137ac1cb27215e25e261c82f116bdbf3" exitCode=0 Sep 08 09:18:03.057301 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:18:03.056842 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" event={"ID":"3606fbcb-b7cb-4c38-b18a-581f6bec472f","Type":"ContainerDied","Data":"3f3acdb2552cc4d32018b9ae293526cd137ac1cb27215e25e261c82f116bdbf3"} Sep 08 09:18:04.066606 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:18:04.066559 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" event={"ID":"3606fbcb-b7cb-4c38-b18a-581f6bec472f","Type":"ContainerStarted","Data":"c184c48a9474dfaf640ae100fae00d34873e56fcd4fda3d70e5fae8790d070b6"} Sep 08 09:18:04.067049 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:18:04.066635 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" event={"ID":"3606fbcb-b7cb-4c38-b18a-581f6bec472f","Type":"ContainerStarted","Data":"69a450ece06cb93c9bd3882ca8bd5dc664485db34cf348a44cd810acc260b4e8"} Sep 08 09:18:04.067049 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:18:04.066659 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" event={"ID":"3606fbcb-b7cb-4c38-b18a-581f6bec472f","Type":"ContainerStarted","Data":"ae6f07e4e0c64af01f1f5796f3a2261f61d0fb0033c3f2dce00357ae2b39833c"} Sep 08 09:18:04.088745 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:18:04.088688 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" podStartSLOduration=32.088671415 podStartE2EDuration="32.088671415s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 09:17:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 09:18:04.086792677 +0000 UTC m=+1830.594899446" watchObservedRunningTime="2026-09-08 09:18:04.088671415 +0000 UTC m=+1830.596778182" Sep 08 09:18:06.694527 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:18:06.694486 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" Sep 08 09:18:06.695037 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:18:06.694552 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" Sep 08 09:18:06.695037 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:18:06.694571 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" Sep 08 09:18:06.696517 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:18:06.696452 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" podUID="3606fbcb-b7cb-4c38-b18a-581f6bec472f" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.32:8443: connect: connection refused" Sep 08 09:18:06.696517 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:18:06.696477 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" podUID="3606fbcb-b7cb-4c38-b18a-581f6bec472f" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.32:8816: connect: connection refused" Sep 08 09:18:06.696689 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:18:06.696486 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" podUID="3606fbcb-b7cb-4c38-b18a-581f6bec472f" containerName="online" probeResult="failure" output="Get \"https://10.132.0.32:6567/health\": dial tcp 10.132.0.32:6567: connect: connection refused" Sep 08 09:18:09.694070 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:18:09.694020 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" podUID="3606fbcb-b7cb-4c38-b18a-581f6bec472f" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.32:8443: connect: connection refused" Sep 08 09:18:09.694515 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:18:09.694070 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" podUID="3606fbcb-b7cb-4c38-b18a-581f6bec472f" containerName="online" probeResult="failure" output="Get \"https://10.132.0.32:6567/health\": dial tcp 10.132.0.32:6567: connect: connection refused" Sep 08 09:18:09.694515 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:18:09.694022 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" podUID="3606fbcb-b7cb-4c38-b18a-581f6bec472f" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.32:8816: connect: connection refused" Sep 08 09:18:12.694459 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:18:12.694424 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" Sep 08 09:18:12.694873 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:18:12.694856 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" Sep 08 09:18:12.694924 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:18:12.694885 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" Sep 08 09:18:12.695149 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:18:12.695129 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" Sep 08 09:18:13.121237 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:18:13.121208 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" Sep 08 09:18:13.121777 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:18:13.121747 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" Sep 08 09:18:13.693919 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:18:13.693884 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" Sep 08 09:18:13.697111 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:18:13.697080 2554 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" podUID="3606fbcb-b7cb-4c38-b18a-581f6bec472f" containerName="online" probeResult="failure" output="Get \"https://10.132.0.32:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 08 09:18:16.443633 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:18:16.443605 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" Sep 08 09:18:17.302844 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:18:17.302811 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" Sep 08 09:19:14.699165 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:14.699077 2554 prober.go:120] "Probe failed" probeType="Liveness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" podUID="3606fbcb-b7cb-4c38-b18a-581f6bec472f" containerName="online" probeResult="failure" output="Get \"https://10.132.0.32:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 08 09:19:18.922164 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:18.922131 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt"] Sep 08 09:19:18.922649 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:18.922364 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" podUID="3606fbcb-b7cb-4c38-b18a-581f6bec472f" containerName="online" containerID="cri-o://ae6f07e4e0c64af01f1f5796f3a2261f61d0fb0033c3f2dce00357ae2b39833c" gracePeriod=30 Sep 08 09:19:18.922649 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:18.922415 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" podUID="3606fbcb-b7cb-4c38-b18a-581f6bec472f" containerName="offline" containerID="cri-o://69a450ece06cb93c9bd3882ca8bd5dc664485db34cf348a44cd810acc260b4e8" gracePeriod=30 Sep 08 09:19:18.922649 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:18.922454 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" podUID="3606fbcb-b7cb-4c38-b18a-581f6bec472f" containerName="ui" containerID="cri-o://c184c48a9474dfaf640ae100fae00d34873e56fcd4fda3d70e5fae8790d070b6" gracePeriod=30 Sep 08 09:19:19.090306 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:19.090272 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn"] Sep 08 09:19:19.090569 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:19.090519 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" podUID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerName="registry" containerID="cri-o://574d43311c11830e649ede911310a7194d20b920ddbc853dfca2e38bd640d075" gracePeriod=30 Sep 08 09:19:19.090569 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:19.090558 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" podUID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerName="offline" containerID="cri-o://534ea69c54e683f1a817d9ddc5d8ee83cb370cbf3211f4223ba39837352832ec" gracePeriod=30 Sep 08 09:19:19.090729 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:19.090609 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" podUID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerName="online" containerID="cri-o://44f74f38fc80a298e67899dd4a5fa4c39fd2e33c947c4354f205d1462d4c8f41" gracePeriod=30 Sep 08 09:19:19.090831 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:19.090806 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" podUID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerName="ui" containerID="cri-o://e4b76f6e0d2cdbb0f96db13738ea2bd3b7168117c0b572db7c9e8673cfe3d075" gracePeriod=30 Sep 08 09:19:20.147180 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:20.147083 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" podUID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.31:8443: connect: connection refused" Sep 08 09:19:20.421472 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:20.421384 2554 generic.go:356] "Generic (PLEG): container finished" podID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerID="e4b76f6e0d2cdbb0f96db13738ea2bd3b7168117c0b572db7c9e8673cfe3d075" exitCode=0 Sep 08 09:19:20.421472 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:20.421410 2554 generic.go:356] "Generic (PLEG): container finished" podID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerID="44f74f38fc80a298e67899dd4a5fa4c39fd2e33c947c4354f205d1462d4c8f41" exitCode=0 Sep 08 09:19:20.421472 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:20.421456 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" event={"ID":"52bcc27d-3095-424f-99f6-64f8d7dae87b","Type":"ContainerDied","Data":"e4b76f6e0d2cdbb0f96db13738ea2bd3b7168117c0b572db7c9e8673cfe3d075"} Sep 08 09:19:20.421708 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:20.421495 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" event={"ID":"52bcc27d-3095-424f-99f6-64f8d7dae87b","Type":"ContainerDied","Data":"44f74f38fc80a298e67899dd4a5fa4c39fd2e33c947c4354f205d1462d4c8f41"} Sep 08 09:19:20.423905 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:20.423883 2554 generic.go:356] "Generic (PLEG): container finished" podID="3606fbcb-b7cb-4c38-b18a-581f6bec472f" containerID="c184c48a9474dfaf640ae100fae00d34873e56fcd4fda3d70e5fae8790d070b6" exitCode=0 Sep 08 09:19:20.423905 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:20.423901 2554 generic.go:356] "Generic (PLEG): container finished" podID="3606fbcb-b7cb-4c38-b18a-581f6bec472f" containerID="ae6f07e4e0c64af01f1f5796f3a2261f61d0fb0033c3f2dce00357ae2b39833c" exitCode=0 Sep 08 09:19:20.424065 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:20.423955 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" event={"ID":"3606fbcb-b7cb-4c38-b18a-581f6bec472f","Type":"ContainerDied","Data":"c184c48a9474dfaf640ae100fae00d34873e56fcd4fda3d70e5fae8790d070b6"} Sep 08 09:19:20.424065 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:20.423995 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" event={"ID":"3606fbcb-b7cb-4c38-b18a-581f6bec472f","Type":"ContainerDied","Data":"ae6f07e4e0c64af01f1f5796f3a2261f61d0fb0033c3f2dce00357ae2b39833c"} Sep 08 09:19:20.643897 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:20.643862 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" podUID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerName="online" probeResult="failure" output="Get \"https://10.132.0.31:6567/health\": dial tcp 10.132.0.31:6567: connect: connection refused" Sep 08 09:19:22.695332 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:22.695293 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" podUID="3606fbcb-b7cb-4c38-b18a-581f6bec472f" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.32:8443: connect: connection refused" Sep 08 09:19:26.444818 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:26.444776 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" podUID="3606fbcb-b7cb-4c38-b18a-581f6bec472f" containerName="online" probeResult="failure" output="Get \"https://10.132.0.32:6567/health\": dial tcp 10.132.0.32:6567: connect: connection refused" Sep 08 09:19:30.146933 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:30.146891 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" podUID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.31:8443: connect: connection refused" Sep 08 09:19:30.643942 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:30.643900 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" podUID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerName="online" probeResult="failure" output="Get \"https://10.132.0.31:6567/health\": dial tcp 10.132.0.31:6567: connect: connection refused" Sep 08 09:19:32.695327 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:32.695284 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" podUID="3606fbcb-b7cb-4c38-b18a-581f6bec472f" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.32:8443: connect: connection refused" Sep 08 09:19:36.444611 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:36.444558 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" podUID="3606fbcb-b7cb-4c38-b18a-581f6bec472f" containerName="online" probeResult="failure" output="Get \"https://10.132.0.32:6567/health\": dial tcp 10.132.0.32:6567: connect: connection refused" Sep 08 09:19:40.147592 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:40.147543 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" podUID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.31:8443: connect: connection refused" Sep 08 09:19:40.148100 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:40.147670 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:19:40.643199 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:40.643160 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" podUID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerName="online" probeResult="failure" output="Get \"https://10.132.0.31:6567/health\": dial tcp 10.132.0.31:6567: connect: connection refused" Sep 08 09:19:40.643454 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:40.643297 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:19:42.695512 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:42.695469 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" podUID="3606fbcb-b7cb-4c38-b18a-581f6bec472f" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.32:8443: connect: connection refused" Sep 08 09:19:42.696003 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:42.695595 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" Sep 08 09:19:46.444753 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:46.444716 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" podUID="3606fbcb-b7cb-4c38-b18a-581f6bec472f" containerName="online" probeResult="failure" output="Get \"https://10.132.0.32:6567/health\": dial tcp 10.132.0.32:6567: connect: connection refused" Sep 08 09:19:46.445160 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:46.444836 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" Sep 08 09:19:49.551595 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.551568 2554 generic.go:356] "Generic (PLEG): container finished" podID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerID="534ea69c54e683f1a817d9ddc5d8ee83cb370cbf3211f4223ba39837352832ec" exitCode=137 Sep 08 09:19:49.551595 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.551594 2554 generic.go:356] "Generic (PLEG): container finished" podID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerID="574d43311c11830e649ede911310a7194d20b920ddbc853dfca2e38bd640d075" exitCode=137 Sep 08 09:19:49.552059 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.551631 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" event={"ID":"52bcc27d-3095-424f-99f6-64f8d7dae87b","Type":"ContainerDied","Data":"534ea69c54e683f1a817d9ddc5d8ee83cb370cbf3211f4223ba39837352832ec"} Sep 08 09:19:49.552059 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.551675 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" event={"ID":"52bcc27d-3095-424f-99f6-64f8d7dae87b","Type":"ContainerDied","Data":"574d43311c11830e649ede911310a7194d20b920ddbc853dfca2e38bd640d075"} Sep 08 09:19:49.554310 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.554289 2554 generic.go:356] "Generic (PLEG): container finished" podID="3606fbcb-b7cb-4c38-b18a-581f6bec472f" containerID="69a450ece06cb93c9bd3882ca8bd5dc664485db34cf348a44cd810acc260b4e8" exitCode=137 Sep 08 09:19:49.554438 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.554358 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" event={"ID":"3606fbcb-b7cb-4c38-b18a-581f6bec472f","Type":"ContainerDied","Data":"69a450ece06cb93c9bd3882ca8bd5dc664485db34cf348a44cd810acc260b4e8"} Sep 08 09:19:49.570934 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.570916 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" Sep 08 09:19:49.723591 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.723569 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:19:49.728088 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.728067 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3606fbcb-b7cb-4c38-b18a-581f6bec472f-ui-tls\") pod \"3606fbcb-b7cb-4c38-b18a-581f6bec472f\" (UID: \"3606fbcb-b7cb-4c38-b18a-581f6bec472f\") " Sep 08 09:19:49.728236 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.728109 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3606fbcb-b7cb-4c38-b18a-581f6bec472f-online-tls\") pod \"3606fbcb-b7cb-4c38-b18a-581f6bec472f\" (UID: \"3606fbcb-b7cb-4c38-b18a-581f6bec472f\") " Sep 08 09:19:49.728236 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.728171 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3606fbcb-b7cb-4c38-b18a-581f6bec472f-feast-data\") pod \"3606fbcb-b7cb-4c38-b18a-581f6bec472f\" (UID: \"3606fbcb-b7cb-4c38-b18a-581f6bec472f\") " Sep 08 09:19:49.728236 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.728189 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/3606fbcb-b7cb-4c38-b18a-581f6bec472f-registry-tls\") pod \"3606fbcb-b7cb-4c38-b18a-581f6bec472f\" (UID: \"3606fbcb-b7cb-4c38-b18a-581f6bec472f\") " Sep 08 09:19:49.728236 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.728230 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9wh9k\" (UniqueName: \"kubernetes.io/projected/3606fbcb-b7cb-4c38-b18a-581f6bec472f-kube-api-access-9wh9k\") pod \"3606fbcb-b7cb-4c38-b18a-581f6bec472f\" (UID: \"3606fbcb-b7cb-4c38-b18a-581f6bec472f\") " Sep 08 09:19:49.728474 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.728287 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3606fbcb-b7cb-4c38-b18a-581f6bec472f-offline-tls\") pod \"3606fbcb-b7cb-4c38-b18a-581f6bec472f\" (UID: \"3606fbcb-b7cb-4c38-b18a-581f6bec472f\") " Sep 08 09:19:49.728598 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.728575 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3606fbcb-b7cb-4c38-b18a-581f6bec472f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "3606fbcb-b7cb-4c38-b18a-581f6bec472f" (UID: "3606fbcb-b7cb-4c38-b18a-581f6bec472f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 09:19:49.728712 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.728686 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3606fbcb-b7cb-4c38-b18a-581f6bec472f-feast-data" (OuterVolumeSpecName: "feast-data") pod "3606fbcb-b7cb-4c38-b18a-581f6bec472f" (UID: "3606fbcb-b7cb-4c38-b18a-581f6bec472f"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 09:19:49.730374 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.730345 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3606fbcb-b7cb-4c38-b18a-581f6bec472f-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "3606fbcb-b7cb-4c38-b18a-581f6bec472f" (UID: "3606fbcb-b7cb-4c38-b18a-581f6bec472f"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 09:19:49.730480 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.730383 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3606fbcb-b7cb-4c38-b18a-581f6bec472f-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "3606fbcb-b7cb-4c38-b18a-581f6bec472f" (UID: "3606fbcb-b7cb-4c38-b18a-581f6bec472f"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 09:19:49.730480 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.730413 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3606fbcb-b7cb-4c38-b18a-581f6bec472f-online-tls" (OuterVolumeSpecName: "online-tls") pod "3606fbcb-b7cb-4c38-b18a-581f6bec472f" (UID: "3606fbcb-b7cb-4c38-b18a-581f6bec472f"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 09:19:49.730480 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.730423 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3606fbcb-b7cb-4c38-b18a-581f6bec472f-kube-api-access-9wh9k" (OuterVolumeSpecName: "kube-api-access-9wh9k") pod "3606fbcb-b7cb-4c38-b18a-581f6bec472f" (UID: "3606fbcb-b7cb-4c38-b18a-581f6bec472f"). InnerVolumeSpecName "kube-api-access-9wh9k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 09:19:49.829337 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.829305 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/52bcc27d-3095-424f-99f6-64f8d7dae87b-registry-tls\") pod \"52bcc27d-3095-424f-99f6-64f8d7dae87b\" (UID: \"52bcc27d-3095-424f-99f6-64f8d7dae87b\") " Sep 08 09:19:49.829337 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.829339 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/52bcc27d-3095-424f-99f6-64f8d7dae87b-online-tls\") pod \"52bcc27d-3095-424f-99f6-64f8d7dae87b\" (UID: \"52bcc27d-3095-424f-99f6-64f8d7dae87b\") " Sep 08 09:19:49.829556 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.829370 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/52bcc27d-3095-424f-99f6-64f8d7dae87b-ui-tls\") pod \"52bcc27d-3095-424f-99f6-64f8d7dae87b\" (UID: \"52bcc27d-3095-424f-99f6-64f8d7dae87b\") " Sep 08 09:19:49.829556 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.829393 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/52bcc27d-3095-424f-99f6-64f8d7dae87b-offline-tls\") pod \"52bcc27d-3095-424f-99f6-64f8d7dae87b\" (UID: \"52bcc27d-3095-424f-99f6-64f8d7dae87b\") " Sep 08 09:19:49.829556 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.829431 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ddcsq\" (UniqueName: \"kubernetes.io/projected/52bcc27d-3095-424f-99f6-64f8d7dae87b-kube-api-access-ddcsq\") pod \"52bcc27d-3095-424f-99f6-64f8d7dae87b\" (UID: \"52bcc27d-3095-424f-99f6-64f8d7dae87b\") " Sep 08 09:19:49.829556 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.829452 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/52bcc27d-3095-424f-99f6-64f8d7dae87b-feast-data\") pod \"52bcc27d-3095-424f-99f6-64f8d7dae87b\" (UID: \"52bcc27d-3095-424f-99f6-64f8d7dae87b\") " Sep 08 09:19:49.830001 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.829633 2554 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3606fbcb-b7cb-4c38-b18a-581f6bec472f-online-tls\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 09:19:49.830001 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.829650 2554 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3606fbcb-b7cb-4c38-b18a-581f6bec472f-feast-data\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 09:19:49.830001 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.829664 2554 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/3606fbcb-b7cb-4c38-b18a-581f6bec472f-registry-tls\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 09:19:49.830001 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.829678 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9wh9k\" (UniqueName: \"kubernetes.io/projected/3606fbcb-b7cb-4c38-b18a-581f6bec472f-kube-api-access-9wh9k\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 09:19:49.830001 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.829692 2554 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3606fbcb-b7cb-4c38-b18a-581f6bec472f-offline-tls\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 09:19:49.830001 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.829705 2554 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3606fbcb-b7cb-4c38-b18a-581f6bec472f-ui-tls\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 09:19:49.830311 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.830108 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52bcc27d-3095-424f-99f6-64f8d7dae87b-feast-data" (OuterVolumeSpecName: "feast-data") pod "52bcc27d-3095-424f-99f6-64f8d7dae87b" (UID: "52bcc27d-3095-424f-99f6-64f8d7dae87b"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 09:19:49.831466 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.831444 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52bcc27d-3095-424f-99f6-64f8d7dae87b-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "52bcc27d-3095-424f-99f6-64f8d7dae87b" (UID: "52bcc27d-3095-424f-99f6-64f8d7dae87b"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 09:19:49.831872 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.831856 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52bcc27d-3095-424f-99f6-64f8d7dae87b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "52bcc27d-3095-424f-99f6-64f8d7dae87b" (UID: "52bcc27d-3095-424f-99f6-64f8d7dae87b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 09:19:49.831943 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.831891 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52bcc27d-3095-424f-99f6-64f8d7dae87b-kube-api-access-ddcsq" (OuterVolumeSpecName: "kube-api-access-ddcsq") pod "52bcc27d-3095-424f-99f6-64f8d7dae87b" (UID: "52bcc27d-3095-424f-99f6-64f8d7dae87b"). InnerVolumeSpecName "kube-api-access-ddcsq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 09:19:49.832022 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.831950 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52bcc27d-3095-424f-99f6-64f8d7dae87b-online-tls" (OuterVolumeSpecName: "online-tls") pod "52bcc27d-3095-424f-99f6-64f8d7dae87b" (UID: "52bcc27d-3095-424f-99f6-64f8d7dae87b"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 09:19:49.832022 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.831963 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52bcc27d-3095-424f-99f6-64f8d7dae87b-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "52bcc27d-3095-424f-99f6-64f8d7dae87b" (UID: "52bcc27d-3095-424f-99f6-64f8d7dae87b"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 09:19:49.930279 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.930252 2554 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/52bcc27d-3095-424f-99f6-64f8d7dae87b-registry-tls\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 09:19:49.930279 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.930274 2554 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/52bcc27d-3095-424f-99f6-64f8d7dae87b-online-tls\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 09:19:49.930424 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.930286 2554 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/52bcc27d-3095-424f-99f6-64f8d7dae87b-ui-tls\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 09:19:49.930424 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.930298 2554 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/52bcc27d-3095-424f-99f6-64f8d7dae87b-offline-tls\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 09:19:49.930424 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.930309 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddcsq\" (UniqueName: \"kubernetes.io/projected/52bcc27d-3095-424f-99f6-64f8d7dae87b-kube-api-access-ddcsq\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 09:19:49.930424 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:49.930322 2554 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/52bcc27d-3095-424f-99f6-64f8d7dae87b-feast-data\") on node \"ip-10-0-141-177.ec2.internal\" DevicePath \"\"" Sep 08 09:19:50.561352 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:50.561315 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" event={"ID":"52bcc27d-3095-424f-99f6-64f8d7dae87b","Type":"ContainerDied","Data":"9d98c277e17d462603c3b2fe13a4f5bff1c343d7c9969d34e9036b9dcfd05821"} Sep 08 09:19:50.561352 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:50.561351 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn" Sep 08 09:19:50.561845 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:50.561381 2554 scope.go:117] "RemoveContainer" containerID="e4b76f6e0d2cdbb0f96db13738ea2bd3b7168117c0b572db7c9e8673cfe3d075" Sep 08 09:19:50.564322 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:50.564304 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" Sep 08 09:19:50.564440 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:50.564324 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt" event={"ID":"3606fbcb-b7cb-4c38-b18a-581f6bec472f","Type":"ContainerDied","Data":"32d0ff8a8e0b606e7f389b411e0e53d36425210b22f83d40d3864e4fa4fdf88e"} Sep 08 09:19:50.570506 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:50.570491 2554 scope.go:117] "RemoveContainer" containerID="534ea69c54e683f1a817d9ddc5d8ee83cb370cbf3211f4223ba39837352832ec" Sep 08 09:19:50.577830 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:50.577810 2554 scope.go:117] "RemoveContainer" containerID="44f74f38fc80a298e67899dd4a5fa4c39fd2e33c947c4354f205d1462d4c8f41" Sep 08 09:19:50.580407 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:50.580373 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt"] Sep 08 09:19:50.583651 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:50.583631 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-t2fgt"] Sep 08 09:19:50.588751 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:50.588731 2554 scope.go:117] "RemoveContainer" containerID="574d43311c11830e649ede911310a7194d20b920ddbc853dfca2e38bd640d075" Sep 08 09:19:50.594785 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:50.594762 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn"] Sep 08 09:19:50.596327 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:50.596313 2554 scope.go:117] "RemoveContainer" containerID="27e1957886956331bead3d8e2c6513b03c02b3889bef217d92ba5b8862154b00" Sep 08 09:19:50.598768 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:50.598750 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-fxqbn"] Sep 08 09:19:50.603708 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:50.603689 2554 scope.go:117] "RemoveContainer" containerID="0f3f340eac5eeaed2d403b5dd41276179f0d2b027fb9835b2b23e1e921740a21" Sep 08 09:19:50.610440 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:50.610423 2554 scope.go:117] "RemoveContainer" containerID="c184c48a9474dfaf640ae100fae00d34873e56fcd4fda3d70e5fae8790d070b6" Sep 08 09:19:50.617150 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:50.617134 2554 scope.go:117] "RemoveContainer" containerID="69a450ece06cb93c9bd3882ca8bd5dc664485db34cf348a44cd810acc260b4e8" Sep 08 09:19:50.623926 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:50.623908 2554 scope.go:117] "RemoveContainer" containerID="ae6f07e4e0c64af01f1f5796f3a2261f61d0fb0033c3f2dce00357ae2b39833c" Sep 08 09:19:50.630758 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:50.630741 2554 scope.go:117] "RemoveContainer" containerID="3f3acdb2552cc4d32018b9ae293526cd137ac1cb27215e25e261c82f116bdbf3" Sep 08 09:19:50.637376 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:50.637360 2554 scope.go:117] "RemoveContainer" containerID="6f656d86a9944b6a7bad2a9470e61a6c58a10739e22b0cdee2dab23bf0758fea" Sep 08 09:19:52.033296 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:52.033263 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3606fbcb-b7cb-4c38-b18a-581f6bec472f" path="/var/lib/kubelet/pods/3606fbcb-b7cb-4c38-b18a-581f6bec472f/volumes" Sep 08 09:19:52.033824 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:19:52.033810 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52bcc27d-3095-424f-99f6-64f8d7dae87b" path="/var/lib/kubelet/pods/52bcc27d-3095-424f-99f6-64f8d7dae87b/volumes" Sep 08 09:20:30.008862 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:30.008809 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-b6s4m_1746be3a-25ff-4dc5-b4fb-6075f46391e0/global-pull-secret-syncer/0.log" Sep 08 09:20:30.185918 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:30.185882 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-lmwtf_72a88ec1-2663-4148-9b6f-0b540b3fb77c/konnectivity-agent/0.log" Sep 08 09:20:30.262908 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:30.262821 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-177.ec2.internal_fdf29c3cd2e403aedd9a5d1cadaf161a/haproxy/0.log" Sep 08 09:20:33.537072 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:33.537043 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d8618cc7-6b6a-4145-b71c-595ec40f4feb/alertmanager/0.log" Sep 08 09:20:33.578151 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:33.578118 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d8618cc7-6b6a-4145-b71c-595ec40f4feb/config-reloader/0.log" Sep 08 09:20:33.604620 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:33.604597 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d8618cc7-6b6a-4145-b71c-595ec40f4feb/kube-rbac-proxy-web/0.log" Sep 08 09:20:33.629720 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:33.629703 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d8618cc7-6b6a-4145-b71c-595ec40f4feb/kube-rbac-proxy/0.log" Sep 08 09:20:33.664879 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:33.664856 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d8618cc7-6b6a-4145-b71c-595ec40f4feb/kube-rbac-proxy-metric/0.log" Sep 08 09:20:33.697047 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:33.697026 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d8618cc7-6b6a-4145-b71c-595ec40f4feb/prom-label-proxy/0.log" Sep 08 09:20:33.726059 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:33.726031 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d8618cc7-6b6a-4145-b71c-595ec40f4feb/init-config-reloader/0.log" Sep 08 09:20:33.816158 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:33.816074 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-hqrm8_925bdd29-aa93-45e4-a301-e74835b7f114/kube-state-metrics/0.log" Sep 08 09:20:33.855273 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:33.855252 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-hqrm8_925bdd29-aa93-45e4-a301-e74835b7f114/kube-rbac-proxy-main/0.log" Sep 08 09:20:33.891376 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:33.891352 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-hqrm8_925bdd29-aa93-45e4-a301-e74835b7f114/kube-rbac-proxy-self/0.log" Sep 08 09:20:33.958949 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:33.958924 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-f4r8r_c7041779-779f-44e5-a529-85744578a714/monitoring-plugin/0.log" Sep 08 09:20:34.107793 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:34.107706 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nlk9v_a2fe8108-3de7-48bb-b768-117699ca6414/node-exporter/0.log" Sep 08 09:20:34.147839 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:34.147804 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nlk9v_a2fe8108-3de7-48bb-b768-117699ca6414/kube-rbac-proxy/0.log" Sep 08 09:20:34.180286 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:34.180265 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nlk9v_a2fe8108-3de7-48bb-b768-117699ca6414/init-textfile/0.log" Sep 08 09:20:34.309959 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:34.309932 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-zb8rb_cb167784-18d1-42da-9020-85970a1e6fdd/kube-rbac-proxy-main/0.log" Sep 08 09:20:34.341013 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:34.340967 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-zb8rb_cb167784-18d1-42da-9020-85970a1e6fdd/kube-rbac-proxy-self/0.log" Sep 08 09:20:34.368754 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:34.368698 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-zb8rb_cb167784-18d1-42da-9020-85970a1e6fdd/openshift-state-metrics/0.log" Sep 08 09:20:34.413918 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:34.413897 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c07b7006-ab2f-4f85-bc2b-3427edcc0923/prometheus/0.log" Sep 08 09:20:34.434075 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:34.434046 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c07b7006-ab2f-4f85-bc2b-3427edcc0923/config-reloader/0.log" Sep 08 09:20:34.462586 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:34.462568 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c07b7006-ab2f-4f85-bc2b-3427edcc0923/thanos-sidecar/0.log" Sep 08 09:20:34.483472 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:34.483441 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c07b7006-ab2f-4f85-bc2b-3427edcc0923/kube-rbac-proxy-web/0.log" Sep 08 09:20:34.510882 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:34.510861 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c07b7006-ab2f-4f85-bc2b-3427edcc0923/kube-rbac-proxy/0.log" Sep 08 09:20:34.532829 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:34.532810 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c07b7006-ab2f-4f85-bc2b-3427edcc0923/kube-rbac-proxy-thanos/0.log" Sep 08 09:20:34.555413 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:34.555395 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c07b7006-ab2f-4f85-bc2b-3427edcc0923/init-config-reloader/0.log" Sep 08 09:20:34.586214 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:34.586184 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-slgg5_1aa1093f-f3ec-4854-81f6-a23ce51ff061/prometheus-operator/0.log" Sep 08 09:20:34.604587 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:34.604569 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-slgg5_1aa1093f-f3ec-4854-81f6-a23ce51ff061/kube-rbac-proxy/0.log" Sep 08 09:20:34.630349 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:34.630297 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-gpzzn_caf19724-9691-48b9-8c53-c5c908cd8bdc/prometheus-operator-admission-webhook/0.log" Sep 08 09:20:34.655164 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:34.655146 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-76497f5b48-g9v24_9afa9c3f-a12a-4e54-867d-0ff38b8eafa8/telemeter-client/0.log" Sep 08 09:20:34.675423 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:34.675403 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-76497f5b48-g9v24_9afa9c3f-a12a-4e54-867d-0ff38b8eafa8/reload/0.log" Sep 08 09:20:34.697695 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:34.697673 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-76497f5b48-g9v24_9afa9c3f-a12a-4e54-867d-0ff38b8eafa8/kube-rbac-proxy/0.log" Sep 08 09:20:36.197770 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:36.197742 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-kcwgm_242ea579-b683-4fe7-a860-89724bce185b/networking-console-plugin/0.log" Sep 08 09:20:36.633165 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:36.633064 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xnd5g_df2da370-7d8d-414d-b82f-3d857fadd070/console-operator/2.log" Sep 08 09:20:36.641048 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:36.641017 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xnd5g_df2da370-7d8d-414d-b82f-3d857fadd070/console-operator/3.log" Sep 08 09:20:37.704272 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.704240 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-g27c6/perf-node-gather-daemonset-2xth6"] Sep 08 09:20:37.704638 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.704559 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3606fbcb-b7cb-4c38-b18a-581f6bec472f" containerName="feast-init" Sep 08 09:20:37.704638 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.704571 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="3606fbcb-b7cb-4c38-b18a-581f6bec472f" containerName="feast-init" Sep 08 09:20:37.704638 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.704580 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerName="feast-apply" Sep 08 09:20:37.704638 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.704585 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerName="feast-apply" Sep 08 09:20:37.704638 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.704597 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3606fbcb-b7cb-4c38-b18a-581f6bec472f" containerName="feast-apply" Sep 08 09:20:37.704638 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.704603 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="3606fbcb-b7cb-4c38-b18a-581f6bec472f" containerName="feast-apply" Sep 08 09:20:37.704638 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.704610 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerName="offline" Sep 08 09:20:37.704638 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.704615 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerName="offline" Sep 08 09:20:37.704638 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.704621 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3606fbcb-b7cb-4c38-b18a-581f6bec472f" containerName="online" Sep 08 09:20:37.704638 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.704627 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="3606fbcb-b7cb-4c38-b18a-581f6bec472f" containerName="online" Sep 08 09:20:37.704638 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.704634 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerName="ui" Sep 08 09:20:37.704638 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.704639 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerName="ui" Sep 08 09:20:37.705056 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.704647 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3606fbcb-b7cb-4c38-b18a-581f6bec472f" containerName="offline" Sep 08 09:20:37.705056 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.704653 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="3606fbcb-b7cb-4c38-b18a-581f6bec472f" containerName="offline" Sep 08 09:20:37.705056 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.704659 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerName="feast-init" Sep 08 09:20:37.705056 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.704664 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerName="feast-init" Sep 08 09:20:37.705056 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.704670 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3606fbcb-b7cb-4c38-b18a-581f6bec472f" containerName="ui" Sep 08 09:20:37.705056 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.704675 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="3606fbcb-b7cb-4c38-b18a-581f6bec472f" containerName="ui" Sep 08 09:20:37.705056 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.704691 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerName="registry" Sep 08 09:20:37.705056 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.704696 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerName="registry" Sep 08 09:20:37.705056 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.704701 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerName="online" Sep 08 09:20:37.705056 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.704706 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerName="online" Sep 08 09:20:37.705056 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.704750 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="3606fbcb-b7cb-4c38-b18a-581f6bec472f" containerName="offline" Sep 08 09:20:37.705056 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.704758 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerName="online" Sep 08 09:20:37.705056 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.704763 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerName="offline" Sep 08 09:20:37.705056 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.704769 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="3606fbcb-b7cb-4c38-b18a-581f6bec472f" containerName="ui" Sep 08 09:20:37.705056 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.704774 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerName="registry" Sep 08 09:20:37.705056 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.704781 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="52bcc27d-3095-424f-99f6-64f8d7dae87b" containerName="ui" Sep 08 09:20:37.705056 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.704787 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="3606fbcb-b7cb-4c38-b18a-581f6bec472f" containerName="online" Sep 08 09:20:37.709276 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.709253 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g27c6/perf-node-gather-daemonset-2xth6" Sep 08 09:20:37.711531 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.711513 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-g27c6\"/\"openshift-service-ca.crt\"" Sep 08 09:20:37.712305 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.712282 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-g27c6\"/\"kube-root-ca.crt\"" Sep 08 09:20:37.712406 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.712282 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-g27c6\"/\"default-dockercfg-pfqv4\"" Sep 08 09:20:37.714561 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.714540 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g27c6/perf-node-gather-daemonset-2xth6"] Sep 08 09:20:37.812542 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.812515 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/95794832-b173-4964-b1f9-123e8c87e890-proc\") pod \"perf-node-gather-daemonset-2xth6\" (UID: \"95794832-b173-4964-b1f9-123e8c87e890\") " pod="openshift-must-gather-g27c6/perf-node-gather-daemonset-2xth6" Sep 08 09:20:37.812694 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.812549 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/95794832-b173-4964-b1f9-123e8c87e890-podres\") pod \"perf-node-gather-daemonset-2xth6\" (UID: \"95794832-b173-4964-b1f9-123e8c87e890\") " pod="openshift-must-gather-g27c6/perf-node-gather-daemonset-2xth6" Sep 08 09:20:37.812694 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.812576 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/95794832-b173-4964-b1f9-123e8c87e890-lib-modules\") pod \"perf-node-gather-daemonset-2xth6\" (UID: \"95794832-b173-4964-b1f9-123e8c87e890\") " pod="openshift-must-gather-g27c6/perf-node-gather-daemonset-2xth6" Sep 08 09:20:37.812694 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.812664 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/95794832-b173-4964-b1f9-123e8c87e890-sys\") pod \"perf-node-gather-daemonset-2xth6\" (UID: \"95794832-b173-4964-b1f9-123e8c87e890\") " pod="openshift-must-gather-g27c6/perf-node-gather-daemonset-2xth6" Sep 08 09:20:37.812694 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.812690 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c78g6\" (UniqueName: \"kubernetes.io/projected/95794832-b173-4964-b1f9-123e8c87e890-kube-api-access-c78g6\") pod \"perf-node-gather-daemonset-2xth6\" (UID: \"95794832-b173-4964-b1f9-123e8c87e890\") " pod="openshift-must-gather-g27c6/perf-node-gather-daemonset-2xth6" Sep 08 09:20:37.913676 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.913647 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/95794832-b173-4964-b1f9-123e8c87e890-proc\") pod \"perf-node-gather-daemonset-2xth6\" (UID: \"95794832-b173-4964-b1f9-123e8c87e890\") " pod="openshift-must-gather-g27c6/perf-node-gather-daemonset-2xth6" Sep 08 09:20:37.913830 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.913682 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/95794832-b173-4964-b1f9-123e8c87e890-podres\") pod \"perf-node-gather-daemonset-2xth6\" (UID: \"95794832-b173-4964-b1f9-123e8c87e890\") " pod="openshift-must-gather-g27c6/perf-node-gather-daemonset-2xth6" Sep 08 09:20:37.913830 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.913709 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/95794832-b173-4964-b1f9-123e8c87e890-lib-modules\") pod \"perf-node-gather-daemonset-2xth6\" (UID: \"95794832-b173-4964-b1f9-123e8c87e890\") " pod="openshift-must-gather-g27c6/perf-node-gather-daemonset-2xth6" Sep 08 09:20:37.913830 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.913739 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/95794832-b173-4964-b1f9-123e8c87e890-sys\") pod \"perf-node-gather-daemonset-2xth6\" (UID: \"95794832-b173-4964-b1f9-123e8c87e890\") " pod="openshift-must-gather-g27c6/perf-node-gather-daemonset-2xth6" Sep 08 09:20:37.913830 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.913772 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c78g6\" (UniqueName: \"kubernetes.io/projected/95794832-b173-4964-b1f9-123e8c87e890-kube-api-access-c78g6\") pod \"perf-node-gather-daemonset-2xth6\" (UID: \"95794832-b173-4964-b1f9-123e8c87e890\") " pod="openshift-must-gather-g27c6/perf-node-gather-daemonset-2xth6" Sep 08 09:20:37.913830 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.913788 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/95794832-b173-4964-b1f9-123e8c87e890-sys\") pod \"perf-node-gather-daemonset-2xth6\" (UID: \"95794832-b173-4964-b1f9-123e8c87e890\") " pod="openshift-must-gather-g27c6/perf-node-gather-daemonset-2xth6" Sep 08 09:20:37.913830 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.913769 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/95794832-b173-4964-b1f9-123e8c87e890-proc\") pod \"perf-node-gather-daemonset-2xth6\" (UID: \"95794832-b173-4964-b1f9-123e8c87e890\") " pod="openshift-must-gather-g27c6/perf-node-gather-daemonset-2xth6" Sep 08 09:20:37.914069 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.913841 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/95794832-b173-4964-b1f9-123e8c87e890-podres\") pod \"perf-node-gather-daemonset-2xth6\" (UID: \"95794832-b173-4964-b1f9-123e8c87e890\") " pod="openshift-must-gather-g27c6/perf-node-gather-daemonset-2xth6" Sep 08 09:20:37.914069 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.913861 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/95794832-b173-4964-b1f9-123e8c87e890-lib-modules\") pod \"perf-node-gather-daemonset-2xth6\" (UID: \"95794832-b173-4964-b1f9-123e8c87e890\") " pod="openshift-must-gather-g27c6/perf-node-gather-daemonset-2xth6" Sep 08 09:20:37.921204 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:37.921179 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c78g6\" (UniqueName: \"kubernetes.io/projected/95794832-b173-4964-b1f9-123e8c87e890-kube-api-access-c78g6\") pod \"perf-node-gather-daemonset-2xth6\" (UID: \"95794832-b173-4964-b1f9-123e8c87e890\") " pod="openshift-must-gather-g27c6/perf-node-gather-daemonset-2xth6" Sep 08 09:20:38.019961 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:38.019902 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g27c6/perf-node-gather-daemonset-2xth6" Sep 08 09:20:38.291601 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:38.291413 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g27c6/perf-node-gather-daemonset-2xth6"] Sep 08 09:20:38.294394 ip-10-0-141-177 kubenswrapper[2554]: W0908 09:20:38.294364 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod95794832_b173_4964_b1f9_123e8c87e890.slice/crio-664925c56cf38b33347f5fea0cd2b26685b33bd3551e64dad8a82809c8e11ccf WatchSource:0}: Error finding container 664925c56cf38b33347f5fea0cd2b26685b33bd3551e64dad8a82809c8e11ccf: Status 404 returned error can't find the container with id 664925c56cf38b33347f5fea0cd2b26685b33bd3551e64dad8a82809c8e11ccf Sep 08 09:20:38.444056 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:38.444032 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wcgdd_47fa42d5-9f58-489c-8031-4a366e32dfc1/dns/0.log" Sep 08 09:20:38.483386 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:38.483359 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wcgdd_47fa42d5-9f58-489c-8031-4a366e32dfc1/kube-rbac-proxy/0.log" Sep 08 09:20:38.600077 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:38.600008 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lbvgv_a4db69aa-e494-477b-8263-f4c7d54f0d71/dns-node-resolver/0.log" Sep 08 09:20:38.768120 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:38.768090 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g27c6/perf-node-gather-daemonset-2xth6" event={"ID":"95794832-b173-4964-b1f9-123e8c87e890","Type":"ContainerStarted","Data":"7bda7b01422de47540aed1802eb7b344601cc3e860919fbbc0c3df8bbb9d37ee"} Sep 08 09:20:38.768586 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:38.768142 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g27c6/perf-node-gather-daemonset-2xth6" event={"ID":"95794832-b173-4964-b1f9-123e8c87e890","Type":"ContainerStarted","Data":"664925c56cf38b33347f5fea0cd2b26685b33bd3551e64dad8a82809c8e11ccf"} Sep 08 09:20:38.768586 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:38.768169 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-g27c6/perf-node-gather-daemonset-2xth6" Sep 08 09:20:38.785843 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:38.785797 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-g27c6/perf-node-gather-daemonset-2xth6" podStartSLOduration=1.785783251 podStartE2EDuration="1.785783251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 09:20:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 09:20:38.784523847 +0000 UTC m=+1985.292630618" watchObservedRunningTime="2026-09-08 09:20:38.785783251 +0000 UTC m=+1985.293890020" Sep 08 09:20:39.060755 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:39.060727 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8bx7f_01944a8d-4aa3-4e4b-9df4-922d4b99f848/node-ca/0.log" Sep 08 09:20:40.150266 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:40.150234 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lp6dm_923c000a-89fd-4a8f-a8ff-5aa9a32c3509/serve-healthcheck-canary/0.log" Sep 08 09:20:40.553410 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:40.553378 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-blnpx_7555e1e8-2114-4e83-aab0-9ffa8e0a065d/kube-rbac-proxy/0.log" Sep 08 09:20:40.573290 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:40.573262 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-blnpx_7555e1e8-2114-4e83-aab0-9ffa8e0a065d/exporter/0.log" Sep 08 09:20:40.593363 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:40.593343 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-blnpx_7555e1e8-2114-4e83-aab0-9ffa8e0a065d/extractor/0.log" Sep 08 09:20:44.636961 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:44.636929 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-sk6kk_db100362-1340-49e9-b87f-73b3d0d415e5/migrator/0.log" Sep 08 09:20:44.654450 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:44.654429 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-sk6kk_db100362-1340-49e9-b87f-73b3d0d415e5/graceful-termination/0.log" Sep 08 09:20:44.781803 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:44.781776 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-g27c6/perf-node-gather-daemonset-2xth6" Sep 08 09:20:44.944577 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:44.944492 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-hs6rq_bd212df8-59bb-46ae-8a18-3d1eb3cbf305/kube-storage-version-migrator-operator/1.log" Sep 08 09:20:44.945952 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:44.945929 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-hs6rq_bd212df8-59bb-46ae-8a18-3d1eb3cbf305/kube-storage-version-migrator-operator/0.log" Sep 08 09:20:46.117076 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:46.117047 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wl97n_4e49d642-1176-4297-84fd-d0cef9884bff/kube-multus-additional-cni-plugins/0.log" Sep 08 09:20:46.140846 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:46.140817 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wl97n_4e49d642-1176-4297-84fd-d0cef9884bff/egress-router-binary-copy/0.log" Sep 08 09:20:46.160580 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:46.160555 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wl97n_4e49d642-1176-4297-84fd-d0cef9884bff/cni-plugins/0.log" Sep 08 09:20:46.182220 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:46.182194 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wl97n_4e49d642-1176-4297-84fd-d0cef9884bff/bond-cni-plugin/0.log" Sep 08 09:20:46.206487 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:46.206466 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wl97n_4e49d642-1176-4297-84fd-d0cef9884bff/routeoverride-cni/0.log" Sep 08 09:20:46.230072 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:46.230046 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wl97n_4e49d642-1176-4297-84fd-d0cef9884bff/whereabouts-cni-bincopy/0.log" Sep 08 09:20:46.259194 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:46.259141 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wl97n_4e49d642-1176-4297-84fd-d0cef9884bff/whereabouts-cni/0.log" Sep 08 09:20:46.297177 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:46.297155 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-txfv4_a2e75b83-e08b-4eee-a253-1d6e4ac0e521/kube-multus/0.log" Sep 08 09:20:46.321145 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:46.321109 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9xk7m_57716ea8-fa5c-47c2-a186-7a0acbf6ae38/network-metrics-daemon/0.log" Sep 08 09:20:46.338715 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:46.338698 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9xk7m_57716ea8-fa5c-47c2-a186-7a0acbf6ae38/kube-rbac-proxy/0.log" Sep 08 09:20:47.650505 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:47.650477 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xn2lh_04c4d313-ed7e-4fea-bec5-3ed11857a8c4/ovn-controller/0.log" Sep 08 09:20:47.665797 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:47.665774 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xn2lh_04c4d313-ed7e-4fea-bec5-3ed11857a8c4/ovn-acl-logging/0.log" Sep 08 09:20:47.665945 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:47.665928 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xn2lh_04c4d313-ed7e-4fea-bec5-3ed11857a8c4/ovn-acl-logging/1.log" Sep 08 09:20:47.685235 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:47.685215 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xn2lh_04c4d313-ed7e-4fea-bec5-3ed11857a8c4/kube-rbac-proxy-node/0.log" Sep 08 09:20:47.707742 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:47.707720 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xn2lh_04c4d313-ed7e-4fea-bec5-3ed11857a8c4/kube-rbac-proxy-ovn-metrics/0.log" Sep 08 09:20:47.723891 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:47.723874 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xn2lh_04c4d313-ed7e-4fea-bec5-3ed11857a8c4/northd/0.log" Sep 08 09:20:47.741687 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:47.741649 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xn2lh_04c4d313-ed7e-4fea-bec5-3ed11857a8c4/nbdb/0.log" Sep 08 09:20:47.764200 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:47.764178 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xn2lh_04c4d313-ed7e-4fea-bec5-3ed11857a8c4/sbdb/0.log" Sep 08 09:20:47.815828 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:47.815806 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xn2lh_04c4d313-ed7e-4fea-bec5-3ed11857a8c4/ovnkube-controller/0.log" Sep 08 09:20:48.839585 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:48.839499 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-685456655f-4zq87_b81d4647-152e-4310-8b25-4624a76694e8/check-endpoints/0.log" Sep 08 09:20:48.910120 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:48.910098 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-z4smd_5e400e6f-9bee-43f4-b6b1-f75ba65c39fc/network-check-target-container/0.log" Sep 08 09:20:49.750085 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:49.750056 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-nsz5v_ab4bc45b-8015-494b-b088-5f699f96acaa/iptables-alerter/0.log" Sep 08 09:20:50.383813 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:50.383763 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-gxjvt_9f7affce-54c6-4ec7-8380-6cae8e0edd91/tuned/0.log" Sep 08 09:20:52.979594 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:52.979562 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-service-ca-operator_service-ca-operator-859f94c4b-v5vkr_5ee267e6-6baf-499a-9e98-78e38f849123/service-ca-operator/1.log" Sep 08 09:20:52.981326 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:52.981294 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-service-ca-operator_service-ca-operator-859f94c4b-v5vkr_5ee267e6-6baf-499a-9e98-78e38f849123/service-ca-operator/0.log" Sep 08 09:20:53.787747 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:53.787716 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-wkwv8_095842a7-07af-47c9-a386-6a90456cc5c0/csi-driver/0.log" Sep 08 09:20:53.808674 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:53.808651 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-wkwv8_095842a7-07af-47c9-a386-6a90456cc5c0/csi-node-driver-registrar/0.log" Sep 08 09:20:53.833760 ip-10-0-141-177 kubenswrapper[2554]: I0908 09:20:53.833731 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-wkwv8_095842a7-07af-47c9-a386-6a90456cc5c0/csi-liveness-probe/0.log"