Sep 09 06:36:17.637450 ip-10-0-132-137 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 09 06:36:17.637468 ip-10-0-132-137 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 09 06:36:17.637495 ip-10-0-132-137 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 09 06:36:17.637806 ip-10-0-132-137 systemd[1]: Failed to start Kubernetes Kubelet. Sep 09 06:36:27.886482 ip-10-0-132-137 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 09 06:36:27.886500 ip-10-0-132-137 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot a08db279df8d4f5984eaa7b800d99fff -- Sep 09 06:38:20.959545 ip-10-0-132-137 systemd[1]: Starting Kubernetes Kubelet... Sep 09 06:38:21.336742 ip-10-0-132-137 kubenswrapper[2559]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 06:38:21.336742 ip-10-0-132-137 kubenswrapper[2559]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 09 06:38:21.336742 ip-10-0-132-137 kubenswrapper[2559]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 06:38:21.336742 ip-10-0-132-137 kubenswrapper[2559]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 09 06:38:21.336742 ip-10-0-132-137 kubenswrapper[2559]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 06:38:21.337396 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.336827 2559 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 09 06:38:21.340623 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340607 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 06:38:21.340623 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340622 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 06:38:21.340682 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340626 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 06:38:21.340682 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340631 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 06:38:21.340682 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340634 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 06:38:21.340682 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340637 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 06:38:21.340682 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340641 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 06:38:21.340682 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340643 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 06:38:21.340682 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340649 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 06:38:21.340682 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340652 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 06:38:21.340682 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340656 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 06:38:21.340682 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340660 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 06:38:21.340682 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340664 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 06:38:21.340682 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340666 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 06:38:21.340682 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340669 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 06:38:21.340682 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340672 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 06:38:21.340682 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340674 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 06:38:21.340682 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340677 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 06:38:21.340682 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340679 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 06:38:21.340682 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340682 2559 feature_gate.go:328] unrecognized feature gate: Example Sep 09 06:38:21.340682 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340685 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 06:38:21.340682 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340688 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 06:38:21.341163 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340691 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 06:38:21.341163 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340694 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 06:38:21.341163 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340697 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 06:38:21.341163 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340699 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 06:38:21.341163 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340702 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 06:38:21.341163 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340704 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 06:38:21.341163 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340707 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 06:38:21.341163 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340709 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 06:38:21.341163 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340712 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 06:38:21.341163 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340714 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 06:38:21.341163 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340717 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 06:38:21.341163 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340719 2559 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 06:38:21.341163 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340722 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 06:38:21.341163 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340724 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 06:38:21.341163 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340727 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 06:38:21.341163 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340730 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 06:38:21.341163 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340732 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 06:38:21.341163 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340736 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 06:38:21.341163 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340740 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 06:38:21.341634 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340743 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 06:38:21.341634 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340747 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 06:38:21.341634 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340751 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 06:38:21.341634 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340754 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 06:38:21.341634 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340757 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 06:38:21.341634 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340759 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 06:38:21.341634 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340762 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 06:38:21.341634 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340764 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 06:38:21.341634 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340767 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 06:38:21.341634 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340769 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 06:38:21.341634 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340772 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 06:38:21.341634 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340774 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 06:38:21.341634 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340778 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 06:38:21.341634 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340781 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 06:38:21.341634 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340783 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 06:38:21.341634 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340786 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 06:38:21.341634 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340789 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 06:38:21.341634 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340791 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 06:38:21.341634 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340794 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 06:38:21.342091 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340797 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 06:38:21.342091 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340799 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 06:38:21.342091 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340802 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 06:38:21.342091 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340804 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 06:38:21.342091 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340807 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 06:38:21.342091 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340809 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 06:38:21.342091 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340812 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 06:38:21.342091 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340814 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 06:38:21.342091 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340817 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 06:38:21.342091 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340819 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 06:38:21.342091 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340822 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 06:38:21.342091 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340825 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 06:38:21.342091 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340828 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 06:38:21.342091 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340830 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 06:38:21.342091 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340833 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 06:38:21.342091 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340835 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 06:38:21.342091 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340838 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 06:38:21.342091 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340841 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 06:38:21.342091 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340844 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 06:38:21.342091 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340847 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 06:38:21.342599 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340849 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 06:38:21.342599 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340852 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 06:38:21.342599 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340854 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 06:38:21.342599 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340857 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 06:38:21.342599 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340859 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 06:38:21.342599 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340862 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 06:38:21.342599 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340865 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 06:38:21.342599 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340869 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 06:38:21.342599 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340872 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 06:38:21.342599 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340874 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 06:38:21.342599 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340877 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 06:38:21.342599 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340879 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 06:38:21.342599 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340882 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 06:38:21.342599 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340884 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 06:38:21.342599 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340887 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 06:38:21.342599 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340890 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 06:38:21.342599 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340892 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 06:38:21.342599 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340894 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 06:38:21.342599 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.340897 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 06:38:21.342599 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341571 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 06:38:21.343083 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341577 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 06:38:21.343083 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341580 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 06:38:21.343083 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341583 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 06:38:21.343083 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341586 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 06:38:21.343083 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341589 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 06:38:21.343083 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341591 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 06:38:21.343083 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341594 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 06:38:21.343083 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341597 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 06:38:21.343083 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341599 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 06:38:21.343083 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341602 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 06:38:21.343083 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341604 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 06:38:21.343083 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341607 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 06:38:21.343083 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341609 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 06:38:21.343083 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341612 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 06:38:21.343083 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341616 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 06:38:21.343083 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341620 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 06:38:21.343083 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341623 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 06:38:21.343083 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341628 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 06:38:21.343083 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341631 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 06:38:21.343593 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341634 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 06:38:21.343593 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341637 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 06:38:21.343593 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341640 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 06:38:21.343593 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341643 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 06:38:21.343593 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341645 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 06:38:21.343593 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341648 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 06:38:21.343593 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341650 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 06:38:21.343593 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341653 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 06:38:21.343593 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341656 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 06:38:21.343593 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341658 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 06:38:21.343593 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341661 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 06:38:21.343593 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341664 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 06:38:21.343593 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341666 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 06:38:21.343593 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341670 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 06:38:21.343593 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341673 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 06:38:21.343593 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341676 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 06:38:21.343593 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341679 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 06:38:21.343593 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341682 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 06:38:21.343593 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341684 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 06:38:21.343593 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341687 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 06:38:21.344092 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341690 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 06:38:21.344092 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341692 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 06:38:21.344092 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341695 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 06:38:21.344092 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341697 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 06:38:21.344092 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341700 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 06:38:21.344092 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341702 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 06:38:21.344092 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341705 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 06:38:21.344092 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341707 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 06:38:21.344092 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341710 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 06:38:21.344092 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341712 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 06:38:21.344092 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341715 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 06:38:21.344092 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341717 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 06:38:21.344092 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341721 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 06:38:21.344092 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341724 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 06:38:21.344092 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341726 2559 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 06:38:21.344092 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341729 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 06:38:21.344092 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341731 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 06:38:21.344092 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341734 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 06:38:21.344092 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341736 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 06:38:21.344572 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341739 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 06:38:21.344572 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341741 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 06:38:21.344572 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341744 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 06:38:21.344572 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341746 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 06:38:21.344572 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341749 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 06:38:21.344572 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341751 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 06:38:21.344572 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341754 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 06:38:21.344572 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341756 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 06:38:21.344572 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341759 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 06:38:21.344572 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341761 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 06:38:21.344572 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341763 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 06:38:21.344572 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341766 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 06:38:21.344572 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341771 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 06:38:21.344572 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341774 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 06:38:21.344572 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341776 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 06:38:21.344572 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341780 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 06:38:21.344572 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341783 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 06:38:21.344572 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341785 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 06:38:21.344572 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341788 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 06:38:21.344572 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341790 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 06:38:21.345084 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341793 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 06:38:21.345084 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341796 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 06:38:21.345084 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341798 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 06:38:21.345084 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341801 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 06:38:21.345084 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341803 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 06:38:21.345084 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341806 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 06:38:21.345084 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341809 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 06:38:21.345084 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341812 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 06:38:21.345084 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341814 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 06:38:21.345084 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341817 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 06:38:21.345084 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341819 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 06:38:21.345084 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341822 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 06:38:21.345084 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341824 2559 feature_gate.go:328] unrecognized feature gate: Example Sep 09 06:38:21.345084 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341827 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 06:38:21.345084 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341830 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 06:38:21.345084 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341833 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 06:38:21.345084 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341836 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 06:38:21.345084 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341838 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 06:38:21.345084 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341841 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 06:38:21.345084 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.341843 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 06:38:21.345587 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342174 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 06:38:21.345587 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342179 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 06:38:21.345587 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342181 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 06:38:21.345587 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342184 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 06:38:21.345587 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342187 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 06:38:21.345587 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342190 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 06:38:21.345587 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342193 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 06:38:21.345587 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342196 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 06:38:21.345587 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342198 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 06:38:21.345587 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342201 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 06:38:21.345587 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342204 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 06:38:21.345587 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342206 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 06:38:21.345587 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342209 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 06:38:21.345587 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342212 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 06:38:21.345587 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342214 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 06:38:21.345587 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342217 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 06:38:21.345587 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342219 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 06:38:21.345587 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342223 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 06:38:21.345587 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342227 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 06:38:21.346053 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342231 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 06:38:21.346053 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342234 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 06:38:21.346053 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342237 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 06:38:21.346053 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342239 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 06:38:21.346053 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342256 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 06:38:21.346053 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342259 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 06:38:21.346053 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342262 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 06:38:21.346053 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342265 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 06:38:21.346053 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342268 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 06:38:21.346053 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342271 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 06:38:21.346053 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342273 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 06:38:21.346053 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342276 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 06:38:21.346053 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342278 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 06:38:21.346053 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342281 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 06:38:21.346053 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342285 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 06:38:21.346053 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342288 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 06:38:21.346053 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342292 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 06:38:21.346053 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342295 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 06:38:21.346053 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342299 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 06:38:21.346543 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342301 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 06:38:21.346543 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342304 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 06:38:21.346543 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342307 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 06:38:21.346543 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342310 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 06:38:21.346543 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342312 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 06:38:21.346543 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342315 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 06:38:21.346543 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342318 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 06:38:21.346543 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342320 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 06:38:21.346543 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342324 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 06:38:21.346543 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342327 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 06:38:21.346543 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342329 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 06:38:21.346543 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342332 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 06:38:21.346543 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342334 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 06:38:21.346543 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342337 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 06:38:21.346543 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342339 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 06:38:21.346543 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342342 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 06:38:21.346543 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342344 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 06:38:21.346543 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342347 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 06:38:21.346543 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342350 2559 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 06:38:21.346543 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342352 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 06:38:21.347032 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342354 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 06:38:21.347032 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342357 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 06:38:21.347032 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342359 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 06:38:21.347032 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342362 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 06:38:21.347032 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342365 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 06:38:21.347032 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342368 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 06:38:21.347032 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342371 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 06:38:21.347032 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342373 2559 feature_gate.go:328] unrecognized feature gate: Example Sep 09 06:38:21.347032 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342377 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 06:38:21.347032 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342379 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 06:38:21.347032 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342382 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 06:38:21.347032 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342386 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 06:38:21.347032 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342390 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 06:38:21.347032 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342392 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 06:38:21.347032 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342395 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 06:38:21.347032 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342397 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 06:38:21.347032 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342400 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 06:38:21.347032 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342402 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 06:38:21.347032 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342405 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 06:38:21.347032 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342407 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 06:38:21.347573 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342410 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 06:38:21.347573 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342412 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 06:38:21.347573 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342415 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 06:38:21.347573 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342417 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 06:38:21.347573 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342420 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 06:38:21.347573 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342422 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 06:38:21.347573 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342425 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 06:38:21.347573 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342427 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 06:38:21.347573 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342430 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 06:38:21.347573 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342432 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 06:38:21.347573 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342435 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 06:38:21.347573 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342437 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 06:38:21.347573 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342440 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 06:38:21.347573 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342442 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 06:38:21.347573 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342445 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 06:38:21.347573 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342449 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 06:38:21.347573 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342451 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 06:38:21.347573 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342454 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 06:38:21.347573 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342456 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 06:38:21.347573 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342459 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 06:38:21.348083 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.342462 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 06:38:21.348083 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342529 2559 flags.go:64] FLAG: --address="0.0.0.0" Sep 09 06:38:21.348083 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342561 2559 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 09 06:38:21.348083 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342569 2559 flags.go:64] FLAG: --anonymous-auth="true" Sep 09 06:38:21.348083 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342574 2559 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 09 06:38:21.348083 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342578 2559 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 09 06:38:21.348083 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342582 2559 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 09 06:38:21.348083 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342586 2559 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 09 06:38:21.348083 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342590 2559 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 09 06:38:21.348083 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342593 2559 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 09 06:38:21.348083 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342596 2559 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 09 06:38:21.348083 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342599 2559 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 09 06:38:21.348083 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342602 2559 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 09 06:38:21.348083 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342606 2559 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 09 06:38:21.348083 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342609 2559 flags.go:64] FLAG: --cgroup-root="" Sep 09 06:38:21.348083 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342612 2559 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 09 06:38:21.348083 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342615 2559 flags.go:64] FLAG: --client-ca-file="" Sep 09 06:38:21.348083 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342617 2559 flags.go:64] FLAG: --cloud-provider="external" Sep 09 06:38:21.348083 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342620 2559 flags.go:64] FLAG: --cluster-dns="[]" Sep 09 06:38:21.348083 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342624 2559 flags.go:64] FLAG: --cluster-domain="" Sep 09 06:38:21.348083 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342627 2559 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 09 06:38:21.348083 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342630 2559 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 09 06:38:21.348083 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342633 2559 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 09 06:38:21.348083 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342637 2559 flags.go:64] FLAG: --container-log-max-files="5" Sep 09 06:38:21.348671 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342640 2559 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 09 06:38:21.348671 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342643 2559 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 09 06:38:21.348671 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342646 2559 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 09 06:38:21.348671 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342652 2559 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 09 06:38:21.348671 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342655 2559 flags.go:64] FLAG: --contention-profiling="false" Sep 09 06:38:21.348671 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342657 2559 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 09 06:38:21.348671 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342660 2559 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 09 06:38:21.348671 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342663 2559 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 09 06:38:21.348671 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342666 2559 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 09 06:38:21.348671 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342670 2559 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 09 06:38:21.348671 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342673 2559 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 09 06:38:21.348671 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342676 2559 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 09 06:38:21.348671 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342679 2559 flags.go:64] FLAG: --enable-load-reader="false" Sep 09 06:38:21.348671 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342682 2559 flags.go:64] FLAG: --enable-server="true" Sep 09 06:38:21.348671 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342685 2559 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 09 06:38:21.348671 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342690 2559 flags.go:64] FLAG: --event-burst="100" Sep 09 06:38:21.348671 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342693 2559 flags.go:64] FLAG: --event-qps="50" Sep 09 06:38:21.348671 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342696 2559 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 09 06:38:21.348671 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342698 2559 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 09 06:38:21.348671 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342701 2559 flags.go:64] FLAG: --eviction-hard="" Sep 09 06:38:21.348671 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342705 2559 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 09 06:38:21.348671 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342708 2559 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 09 06:38:21.348671 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342711 2559 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 09 06:38:21.348671 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342714 2559 flags.go:64] FLAG: --eviction-soft="" Sep 09 06:38:21.348671 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342717 2559 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 09 06:38:21.349299 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342720 2559 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 09 06:38:21.349299 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342723 2559 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 09 06:38:21.349299 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342725 2559 flags.go:64] FLAG: --experimental-mounter-path="" Sep 09 06:38:21.349299 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342728 2559 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 09 06:38:21.349299 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342731 2559 flags.go:64] FLAG: --fail-swap-on="true" Sep 09 06:38:21.349299 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342734 2559 flags.go:64] FLAG: --feature-gates="" Sep 09 06:38:21.349299 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342737 2559 flags.go:64] FLAG: --file-check-frequency="20s" Sep 09 06:38:21.349299 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342740 2559 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 09 06:38:21.349299 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342743 2559 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 09 06:38:21.349299 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342746 2559 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 09 06:38:21.349299 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342750 2559 flags.go:64] FLAG: --healthz-port="10248" Sep 09 06:38:21.349299 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342753 2559 flags.go:64] FLAG: --help="false" Sep 09 06:38:21.349299 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342756 2559 flags.go:64] FLAG: --hostname-override="ip-10-0-132-137.ec2.internal" Sep 09 06:38:21.349299 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342759 2559 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 09 06:38:21.349299 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342762 2559 flags.go:64] FLAG: --http-check-frequency="20s" Sep 09 06:38:21.349299 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342765 2559 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 09 06:38:21.349299 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342769 2559 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 09 06:38:21.349299 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342773 2559 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 09 06:38:21.349299 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342775 2559 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 09 06:38:21.349299 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342778 2559 flags.go:64] FLAG: --image-service-endpoint="" Sep 09 06:38:21.349299 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342781 2559 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 09 06:38:21.349299 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342784 2559 flags.go:64] FLAG: --kube-api-burst="100" Sep 09 06:38:21.349299 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342787 2559 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 09 06:38:21.349962 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342791 2559 flags.go:64] FLAG: --kube-api-qps="50" Sep 09 06:38:21.349962 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342793 2559 flags.go:64] FLAG: --kube-reserved="" Sep 09 06:38:21.349962 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342796 2559 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 09 06:38:21.349962 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342799 2559 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 09 06:38:21.349962 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342802 2559 flags.go:64] FLAG: --kubelet-cgroups="" Sep 09 06:38:21.349962 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342805 2559 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 09 06:38:21.349962 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342808 2559 flags.go:64] FLAG: --lock-file="" Sep 09 06:38:21.349962 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342810 2559 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 09 06:38:21.349962 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342813 2559 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 09 06:38:21.349962 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342816 2559 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 09 06:38:21.349962 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342821 2559 flags.go:64] FLAG: --log-json-split-stream="false" Sep 09 06:38:21.349962 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342824 2559 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 09 06:38:21.349962 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342827 2559 flags.go:64] FLAG: --log-text-split-stream="false" Sep 09 06:38:21.349962 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342830 2559 flags.go:64] FLAG: --logging-format="text" Sep 09 06:38:21.349962 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342832 2559 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 09 06:38:21.349962 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342836 2559 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 09 06:38:21.349962 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342838 2559 flags.go:64] FLAG: --manifest-url="" Sep 09 06:38:21.349962 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342841 2559 flags.go:64] FLAG: --manifest-url-header="" Sep 09 06:38:21.349962 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342845 2559 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 09 06:38:21.349962 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342860 2559 flags.go:64] FLAG: --max-open-files="1000000" Sep 09 06:38:21.349962 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342864 2559 flags.go:64] FLAG: --max-pods="110" Sep 09 06:38:21.349962 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342867 2559 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 09 06:38:21.349962 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342870 2559 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 09 06:38:21.349962 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342873 2559 flags.go:64] FLAG: --memory-manager-policy="None" Sep 09 06:38:21.349962 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342876 2559 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 09 06:38:21.350586 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342879 2559 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 09 06:38:21.350586 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342882 2559 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 09 06:38:21.350586 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342885 2559 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 09 06:38:21.350586 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342891 2559 flags.go:64] FLAG: --node-status-max-images="50" Sep 09 06:38:21.350586 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342894 2559 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 09 06:38:21.350586 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342897 2559 flags.go:64] FLAG: --oom-score-adj="-999" Sep 09 06:38:21.350586 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342900 2559 flags.go:64] FLAG: --pod-cidr="" Sep 09 06:38:21.350586 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342903 2559 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 09 06:38:21.350586 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342909 2559 flags.go:64] FLAG: --pod-manifest-path="" Sep 09 06:38:21.350586 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342912 2559 flags.go:64] FLAG: --pod-max-pids="-1" Sep 09 06:38:21.350586 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342915 2559 flags.go:64] FLAG: --pods-per-core="0" Sep 09 06:38:21.350586 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342918 2559 flags.go:64] FLAG: --port="10250" Sep 09 06:38:21.350586 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342921 2559 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 09 06:38:21.350586 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342924 2559 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-017c682f10463e54e" Sep 09 06:38:21.350586 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342927 2559 flags.go:64] FLAG: --qos-reserved="" Sep 09 06:38:21.350586 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342930 2559 flags.go:64] FLAG: --read-only-port="10255" Sep 09 06:38:21.350586 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342933 2559 flags.go:64] FLAG: --register-node="true" Sep 09 06:38:21.350586 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342936 2559 flags.go:64] FLAG: --register-with-taints="" Sep 09 06:38:21.350586 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342940 2559 flags.go:64] FLAG: --registry-burst="10" Sep 09 06:38:21.350586 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342942 2559 flags.go:64] FLAG: --registry-qps="5" Sep 09 06:38:21.350586 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342945 2559 flags.go:64] FLAG: --reserved-cpus="" Sep 09 06:38:21.350586 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342948 2559 flags.go:64] FLAG: --reserved-memory="" Sep 09 06:38:21.350586 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342951 2559 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 09 06:38:21.350586 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342954 2559 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 09 06:38:21.350586 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342957 2559 flags.go:64] FLAG: --rotate-certificates="false" Sep 09 06:38:21.351187 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342960 2559 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 09 06:38:21.351187 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342962 2559 flags.go:64] FLAG: --runonce="false" Sep 09 06:38:21.351187 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342966 2559 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 09 06:38:21.351187 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342969 2559 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 09 06:38:21.351187 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342972 2559 flags.go:64] FLAG: --seccomp-default="false" Sep 09 06:38:21.351187 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342975 2559 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 09 06:38:21.351187 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342978 2559 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 09 06:38:21.351187 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342980 2559 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 09 06:38:21.351187 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342983 2559 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 09 06:38:21.351187 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342986 2559 flags.go:64] FLAG: --storage-driver-password="root" Sep 09 06:38:21.351187 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342990 2559 flags.go:64] FLAG: --storage-driver-secure="false" Sep 09 06:38:21.351187 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342992 2559 flags.go:64] FLAG: --storage-driver-table="stats" Sep 09 06:38:21.351187 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342995 2559 flags.go:64] FLAG: --storage-driver-user="root" Sep 09 06:38:21.351187 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.342998 2559 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 09 06:38:21.351187 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.343001 2559 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 09 06:38:21.351187 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.343004 2559 flags.go:64] FLAG: --system-cgroups="" Sep 09 06:38:21.351187 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.343007 2559 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 09 06:38:21.351187 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.343012 2559 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 09 06:38:21.351187 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.343015 2559 flags.go:64] FLAG: --tls-cert-file="" Sep 09 06:38:21.351187 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.343017 2559 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 09 06:38:21.351187 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.343021 2559 flags.go:64] FLAG: --tls-min-version="" Sep 09 06:38:21.351187 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.343024 2559 flags.go:64] FLAG: --tls-private-key-file="" Sep 09 06:38:21.351187 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.343026 2559 flags.go:64] FLAG: --topology-manager-policy="none" Sep 09 06:38:21.351187 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.343029 2559 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 09 06:38:21.351187 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.343032 2559 flags.go:64] FLAG: --topology-manager-scope="container" Sep 09 06:38:21.351847 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.343035 2559 flags.go:64] FLAG: --v="2" Sep 09 06:38:21.351847 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.343038 2559 flags.go:64] FLAG: --version="false" Sep 09 06:38:21.351847 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.343042 2559 flags.go:64] FLAG: --vmodule="" Sep 09 06:38:21.351847 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.343045 2559 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 09 06:38:21.351847 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.343048 2559 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 09 06:38:21.351847 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343152 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 06:38:21.351847 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343164 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 06:38:21.351847 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343168 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 06:38:21.351847 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343171 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 06:38:21.351847 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343176 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 06:38:21.351847 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343179 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 06:38:21.351847 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343181 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 06:38:21.351847 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343184 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 06:38:21.351847 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343187 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 06:38:21.351847 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343189 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 06:38:21.351847 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343192 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 06:38:21.351847 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343197 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 06:38:21.351847 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343199 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 06:38:21.351847 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343202 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 06:38:21.351847 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343204 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 06:38:21.352347 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343208 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 06:38:21.352347 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343212 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 06:38:21.352347 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343215 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 06:38:21.352347 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343218 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 06:38:21.352347 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343220 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 06:38:21.352347 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343223 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 06:38:21.352347 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343226 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 06:38:21.352347 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343229 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 06:38:21.352347 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343231 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 06:38:21.352347 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343234 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 06:38:21.352347 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343236 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 06:38:21.352347 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343239 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 06:38:21.352347 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343255 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 06:38:21.352347 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343258 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 06:38:21.352347 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343261 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 06:38:21.352347 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343264 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 06:38:21.352347 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343267 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 06:38:21.352347 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343269 2559 feature_gate.go:328] unrecognized feature gate: Example Sep 09 06:38:21.352347 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343272 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 06:38:21.352347 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343275 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 06:38:21.352822 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343278 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 06:38:21.352822 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343282 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 06:38:21.352822 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343284 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 06:38:21.352822 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343287 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 06:38:21.352822 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343290 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 06:38:21.352822 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343292 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 06:38:21.352822 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343295 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 06:38:21.352822 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343298 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 06:38:21.352822 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343302 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 06:38:21.352822 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343305 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 06:38:21.352822 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343308 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 06:38:21.352822 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343310 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 06:38:21.352822 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343313 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 06:38:21.352822 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343315 2559 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 06:38:21.352822 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343318 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 06:38:21.352822 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343321 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 06:38:21.352822 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343324 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 06:38:21.352822 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343326 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 06:38:21.352822 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343331 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 06:38:21.353319 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343334 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 06:38:21.353319 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343337 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 06:38:21.353319 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343340 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 06:38:21.353319 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343343 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 06:38:21.353319 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343346 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 06:38:21.353319 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343348 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 06:38:21.353319 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343351 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 06:38:21.353319 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343354 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 06:38:21.353319 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343356 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 06:38:21.353319 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343360 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 06:38:21.353319 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343363 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 06:38:21.353319 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343366 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 06:38:21.353319 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343369 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 06:38:21.353319 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343371 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 06:38:21.353319 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343375 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 06:38:21.353319 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343378 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 06:38:21.353319 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343380 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 06:38:21.353319 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343383 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 06:38:21.353319 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343385 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 06:38:21.353785 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343388 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 06:38:21.353785 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343390 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 06:38:21.353785 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343394 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 06:38:21.353785 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343397 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 06:38:21.353785 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343399 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 06:38:21.353785 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343402 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 06:38:21.353785 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343405 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 06:38:21.353785 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343407 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 06:38:21.353785 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343410 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 06:38:21.353785 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343412 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 06:38:21.353785 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343415 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 06:38:21.353785 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343418 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 06:38:21.353785 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343420 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 06:38:21.353785 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343423 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 06:38:21.353785 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343425 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 06:38:21.353785 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343428 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 06:38:21.353785 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343431 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 06:38:21.353785 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343433 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 06:38:21.353785 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343436 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 06:38:21.353785 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343438 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 06:38:21.354282 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343441 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 06:38:21.354282 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343443 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 06:38:21.354282 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343446 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 06:38:21.354282 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343448 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 06:38:21.354282 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343451 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 06:38:21.354282 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.343454 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 06:38:21.354282 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.344041 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 06:38:21.354282 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.350797 2559 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 09 06:38:21.354282 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.350812 2559 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 09 06:38:21.354282 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.351455 2559 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 09 06:38:21.354282 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.351462 2559 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 09 06:38:21.354282 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351516 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 06:38:21.354282 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351521 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 06:38:21.354282 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351525 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 06:38:21.354282 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351527 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 06:38:21.354668 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351530 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 06:38:21.354668 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351533 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 06:38:21.354668 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351538 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 06:38:21.354668 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351541 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 06:38:21.354668 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351545 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 06:38:21.354668 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351548 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 06:38:21.354668 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351551 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 06:38:21.354668 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351554 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 06:38:21.354668 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351557 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 06:38:21.354668 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351560 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 06:38:21.354668 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351563 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 06:38:21.354668 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351566 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 06:38:21.354668 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351569 2559 feature_gate.go:328] unrecognized feature gate: Example Sep 09 06:38:21.354668 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351572 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 06:38:21.354668 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351574 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 06:38:21.354668 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351577 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 06:38:21.354668 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351579 2559 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 06:38:21.354668 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351582 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 06:38:21.354668 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351584 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 06:38:21.355157 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351587 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 06:38:21.355157 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351589 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 06:38:21.355157 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351592 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 06:38:21.355157 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351594 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 06:38:21.355157 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351597 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 06:38:21.355157 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351600 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 06:38:21.355157 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351608 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 06:38:21.355157 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351610 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 06:38:21.355157 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351613 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 06:38:21.355157 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351616 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 06:38:21.355157 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351620 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 06:38:21.355157 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351624 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 06:38:21.355157 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351628 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 06:38:21.355157 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351632 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 06:38:21.355157 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351635 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 06:38:21.355157 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351638 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 06:38:21.355157 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351640 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 06:38:21.355157 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351643 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 06:38:21.355157 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351645 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 06:38:21.355157 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351648 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 06:38:21.355667 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351651 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 06:38:21.355667 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351653 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 06:38:21.355667 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351656 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 06:38:21.355667 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351658 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 06:38:21.355667 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351661 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 06:38:21.355667 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351663 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 06:38:21.355667 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351665 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 06:38:21.355667 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351668 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 06:38:21.355667 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351671 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 06:38:21.355667 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351673 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 06:38:21.355667 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351676 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 06:38:21.355667 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351678 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 06:38:21.355667 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351681 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 06:38:21.355667 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351683 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 06:38:21.355667 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351686 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 06:38:21.355667 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351688 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 06:38:21.355667 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351691 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 06:38:21.355667 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351693 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 06:38:21.355667 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351696 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 06:38:21.356130 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351700 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 06:38:21.356130 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351703 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 06:38:21.356130 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351705 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 06:38:21.356130 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351708 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 06:38:21.356130 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351710 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 06:38:21.356130 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351713 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 06:38:21.356130 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351715 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 06:38:21.356130 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351718 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 06:38:21.356130 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351720 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 06:38:21.356130 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351723 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 06:38:21.356130 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351725 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 06:38:21.356130 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351728 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 06:38:21.356130 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351730 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 06:38:21.356130 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351733 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 06:38:21.356130 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351735 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 06:38:21.356130 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351737 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 06:38:21.356130 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351740 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 06:38:21.356130 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351742 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 06:38:21.356130 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351745 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 06:38:21.356130 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351747 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 06:38:21.356624 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351750 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 06:38:21.356624 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351752 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 06:38:21.356624 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351755 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 06:38:21.356624 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351758 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 06:38:21.356624 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351760 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 06:38:21.356624 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351762 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 06:38:21.356624 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351765 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 06:38:21.356624 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351767 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 06:38:21.356624 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351770 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 06:38:21.356624 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351773 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 06:38:21.356624 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351775 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 06:38:21.356624 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351778 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 06:38:21.356624 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351780 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 06:38:21.356624 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351784 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 06:38:21.356624 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351786 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 06:38:21.356624 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351788 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 06:38:21.356624 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351791 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 06:38:21.357043 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.351796 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 06:38:21.357043 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351898 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 06:38:21.357043 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351906 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 06:38:21.357043 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351909 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 06:38:21.357043 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351911 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 06:38:21.357043 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351914 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 06:38:21.357043 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351916 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 06:38:21.357043 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351919 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 06:38:21.357043 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351921 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 06:38:21.357043 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351923 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 06:38:21.357043 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351926 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 06:38:21.357043 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351928 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 06:38:21.357043 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351931 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 06:38:21.357043 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351933 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 06:38:21.357043 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351936 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 06:38:21.357426 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351938 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 06:38:21.357426 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351941 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 06:38:21.357426 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351943 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 06:38:21.357426 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351946 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 06:38:21.357426 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351948 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 06:38:21.357426 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351950 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 06:38:21.357426 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351953 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 06:38:21.357426 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351956 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 06:38:21.357426 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351958 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 06:38:21.357426 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351962 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 06:38:21.357426 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351966 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 06:38:21.357426 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351969 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 06:38:21.357426 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351972 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 06:38:21.357426 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351975 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 06:38:21.357426 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351978 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 06:38:21.357426 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351981 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 06:38:21.357426 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351983 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 06:38:21.357426 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351986 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 06:38:21.357426 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351988 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 06:38:21.357886 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351991 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 06:38:21.357886 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351993 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 06:38:21.357886 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351996 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 06:38:21.357886 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.351998 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 06:38:21.357886 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352001 2559 feature_gate.go:328] unrecognized feature gate: Example Sep 09 06:38:21.357886 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352003 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 06:38:21.357886 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352005 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 06:38:21.357886 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352008 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 06:38:21.357886 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352010 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 06:38:21.357886 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352013 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 06:38:21.357886 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352015 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 06:38:21.357886 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352018 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 06:38:21.357886 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352020 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 06:38:21.357886 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352022 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 06:38:21.357886 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352025 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 06:38:21.357886 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352027 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 06:38:21.357886 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352030 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 06:38:21.357886 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352032 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 06:38:21.357886 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352036 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 06:38:21.357886 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352039 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 06:38:21.358494 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352042 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 06:38:21.358494 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352045 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 06:38:21.358494 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352047 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 06:38:21.358494 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352050 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 06:38:21.358494 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352053 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 06:38:21.358494 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352055 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 06:38:21.358494 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352058 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 06:38:21.358494 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352060 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 06:38:21.358494 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352063 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 06:38:21.358494 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352066 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 06:38:21.358494 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352068 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 06:38:21.358494 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352071 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 06:38:21.358494 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352073 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 06:38:21.358494 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352076 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 06:38:21.358494 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352078 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 06:38:21.358494 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352081 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 06:38:21.358494 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352083 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 06:38:21.358494 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352086 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 06:38:21.358494 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352088 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 06:38:21.358494 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352091 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 06:38:21.358972 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352093 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 06:38:21.358972 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352095 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 06:38:21.358972 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352098 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 06:38:21.358972 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352100 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 06:38:21.358972 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352103 2559 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 06:38:21.358972 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352106 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 06:38:21.358972 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352108 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 06:38:21.358972 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352112 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 06:38:21.358972 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352115 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 06:38:21.358972 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352118 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 06:38:21.358972 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352121 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 06:38:21.358972 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352124 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 06:38:21.358972 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352127 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 06:38:21.358972 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352129 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 06:38:21.358972 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352132 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 06:38:21.358972 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352134 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 06:38:21.358972 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352137 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 06:38:21.358972 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352140 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 06:38:21.358972 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352142 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 06:38:21.359431 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352145 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 06:38:21.359431 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352147 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 06:38:21.359431 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352150 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 06:38:21.359431 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352152 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 06:38:21.359431 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352155 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 06:38:21.359431 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352157 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 06:38:21.359431 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:21.352160 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 06:38:21.359431 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.352164 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 06:38:21.359431 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.352294 2559 server.go:962] "Client rotation is on, will bootstrap in background" Sep 09 06:38:21.359431 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.354132 2559 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 09 06:38:21.359431 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.355013 2559 server.go:1019] "Starting client certificate rotation" Sep 09 06:38:21.359431 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.355111 2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 09 06:38:21.359431 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.355173 2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 09 06:38:21.359756 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.357315 2559 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 09 06:38:21.368206 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.368182 2559 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 09 06:38:21.383448 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.383426 2559 log.go:25] "Validated CRI v1 runtime API" Sep 09 06:38:21.386521 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.386509 2559 log.go:25] "Validated CRI v1 image API" Sep 09 06:38:21.387305 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.387289 2559 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 09 06:38:21.387797 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.387781 2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 09 06:38:21.390725 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.390701 2559 fs.go:135] Filesystem UUIDs: map[5d1a9f56-7174-47a7-b2cb-caf08995c91b:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 8f2ba3cf-0722-4070-b084-da2c2cedf59c:/dev/nvme0n1p3] Sep 09 06:38:21.390781 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.390724 2559 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 09 06:38:21.395703 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.395596 2559 manager.go:216] Machine: {Timestamp:2026-09-09 06:38:21.394330749 +0000 UTC m=+0.322556408 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098722 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec288cef7496a04f2d737725e44dfad4 SystemUUID:ec288cef-7496-a04f-2d73-7725e44dfad4 BootID:a08db279-df8d-4f59-84ea-a7b800d99fff Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:65:2d:0c:8b:2b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:65:2d:0c:8b:2b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fe:79:82:ba:c2:cc Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 09 06:38:21.395703 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.395698 2559 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 09 06:38:21.395817 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.395771 2559 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 09 06:38:21.399036 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.399012 2559 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 09 06:38:21.399156 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.399039 2559 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-137.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 09 06:38:21.399680 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.399670 2559 topology_manager.go:138] "Creating topology manager with none policy" Sep 09 06:38:21.399717 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.399683 2559 container_manager_linux.go:309] "Creating device plugin manager" Sep 09 06:38:21.399717 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.399691 2559 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 09 06:38:21.400575 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.400564 2559 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 09 06:38:21.401772 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.401762 2559 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 09 06:38:21.405341 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.405330 2559 state_mem.go:36] "Initialized new in-memory state store" Sep 09 06:38:21.405463 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.405453 2559 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 09 06:38:21.405494 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.405486 2559 kubelet.go:486] "Attempting to sync node with API server" Sep 09 06:38:21.405527 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.405498 2559 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 09 06:38:21.405527 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.405512 2559 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 09 06:38:21.405527 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.405521 2559 kubelet.go:398] "Adding apiserver pod source" Sep 09 06:38:21.405614 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.405530 2559 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 09 06:38:21.405842 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.405829 2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 09 06:38:21.405894 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.405847 2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 09 06:38:21.408052 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.408036 2559 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 09 06:38:21.410195 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.410180 2559 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 09 06:38:21.410286 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.410228 2559 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 09 06:38:21.410935 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.410922 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 09 06:38:21.411003 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.410945 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 09 06:38:21.411003 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.410956 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 09 06:38:21.411003 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.410964 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 09 06:38:21.411003 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.410975 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 09 06:38:21.411003 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.410983 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 09 06:38:21.411192 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.411079 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 09 06:38:21.411192 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.411088 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 09 06:38:21.411192 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.411097 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 09 06:38:21.411192 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.411120 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 09 06:38:21.411192 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.411138 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 09 06:38:21.411192 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.411157 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 09 06:38:21.412784 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.412773 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 09 06:38:21.412836 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.412787 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 09 06:38:21.416708 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.416689 2559 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-137.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 09 06:38:21.416787 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:21.416768 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-137.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 09 06:38:21.416951 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:21.416912 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 09 06:38:21.418330 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.418317 2559 server.go:1268] "Started kubelet" Sep 09 06:38:21.418424 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.418398 2559 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 09 06:38:21.418463 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.418405 2559 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 09 06:38:21.418503 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.418466 2559 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 09 06:38:21.419267 ip-10-0-132-137 systemd[1]: Started Kubernetes Kubelet. Sep 09 06:38:21.419507 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.419493 2559 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 09 06:38:21.420334 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.420310 2559 server.go:310] "Adding debug handlers to kubelet server" Sep 09 06:38:21.422590 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:21.422567 2559 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 09 06:38:21.423886 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.423868 2559 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 09 06:38:21.423958 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.423876 2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 09 06:38:21.424018 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:21.423991 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-137.ec2.internal\" not found" Sep 09 06:38:21.424450 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:21.423132 2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-137.ec2.internal.18d393b961fadfd0 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-137.ec2.internal,UID:ip-10-0-132-137.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-137.ec2.internal,},FirstTimestamp:2026-09-09 06:38:21.41828296 +0000 UTC m=+0.346508619,LastTimestamp:2026-09-09 06:38:21.41828296 +0000 UTC m=+0.346508619,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-137.ec2.internal,}" Sep 09 06:38:21.424541 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.424490 2559 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 09 06:38:21.424541 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.424519 2559 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 09 06:38:21.424637 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.424617 2559 reconstruct.go:97] "Volume reconstruction finished" Sep 09 06:38:21.424637 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.424624 2559 reconciler.go:29] "Reconciler: start to sync state" Sep 09 06:38:21.424735 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.424643 2559 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 09 06:38:21.424735 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.424708 2559 factory.go:55] Registering systemd factory Sep 09 06:38:21.424735 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.424726 2559 factory.go:223] Registration of the systemd container factory successfully Sep 09 06:38:21.424953 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.424927 2559 factory.go:153] Registering CRI-O factory Sep 09 06:38:21.425009 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.424954 2559 factory.go:223] Registration of the crio container factory successfully Sep 09 06:38:21.425061 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.425007 2559 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 09 06:38:21.425061 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.425036 2559 factory.go:103] Registering Raw factory Sep 09 06:38:21.425061 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.425051 2559 manager.go:1196] Started watching for new ooms in manager Sep 09 06:38:21.425571 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.425558 2559 manager.go:349] Starting recovery of all containers Sep 09 06:38:21.425981 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:21.425959 2559 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-137.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 09 06:38:21.426117 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:21.426096 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 09 06:38:21.430801 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.430777 2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mt97k" Sep 09 06:38:21.431222 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.431176 2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 09 06:38:21.436802 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.436783 2559 manager.go:354] Recovery completed Sep 09 06:38:21.438577 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.438551 2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mt97k" Sep 09 06:38:21.441849 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.441834 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 06:38:21.443736 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.443631 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-137.ec2.internal" event="NodeHasSufficientMemory" Sep 09 06:38:21.443798 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.443747 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-137.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 06:38:21.443798 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.443757 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-137.ec2.internal" event="NodeHasSufficientPID" Sep 09 06:38:21.444295 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.444279 2559 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 09 06:38:21.444295 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.444293 2559 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 09 06:38:21.444295 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.444294 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 06:38:21.444450 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.444307 2559 state_mem.go:36] "Initialized new in-memory state store" Sep 09 06:38:21.445073 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.445052 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-137.ec2.internal" event="NodeHasSufficientMemory" Sep 09 06:38:21.445139 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.445085 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-137.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 06:38:21.445139 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.445094 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-137.ec2.internal" event="NodeHasSufficientPID" Sep 09 06:38:21.446229 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.446212 2559 policy_none.go:49] "None policy: Start" Sep 09 06:38:21.446317 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.446232 2559 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 09 06:38:21.446317 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.446259 2559 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 09 06:38:21.448313 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.448300 2559 policy_none.go:47] "Start" Sep 09 06:38:21.471858 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.471843 2559 manager.go:337] "Starting Device Plugin manager" Sep 09 06:38:21.596674 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:21.471883 2559 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 09 06:38:21.596674 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.471895 2559 server.go:87] "Starting device plugin registration server" Sep 09 06:38:21.596674 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.472152 2559 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 09 06:38:21.596674 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.472163 2559 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 09 06:38:21.596674 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.472517 2559 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 09 06:38:21.596674 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.472583 2559 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 09 06:38:21.596674 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.472591 2559 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 09 06:38:21.596674 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:21.474051 2559 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 09 06:38:21.596674 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:21.474101 2559 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-137.ec2.internal\" not found" Sep 09 06:38:21.596674 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.491588 2559 container_manager_linux.go:676] "Failed to ensure state" containerName="/system.slice" err="failed to move PID 2599 into the system container \"/system.slice\": " Sep 09 06:38:21.596674 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.572738 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 06:38:21.596674 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.573586 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-137.ec2.internal" event="NodeHasSufficientMemory" Sep 09 06:38:21.596674 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.573614 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-137.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 06:38:21.596674 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.573626 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-137.ec2.internal" event="NodeHasSufficientPID" Sep 09 06:38:21.596674 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.573656 2559 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-137.ec2.internal" Sep 09 06:38:21.596674 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.582460 2559 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-137.ec2.internal" Sep 09 06:38:21.596674 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:21.582477 2559 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-137.ec2.internal\": node \"ip-10-0-132-137.ec2.internal\" not found" Sep 09 06:38:21.602180 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:21.602159 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-137.ec2.internal\" not found" Sep 09 06:38:21.664232 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.664211 2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 09 06:38:21.664348 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.664283 2559 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 09 06:38:21.664348 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.664316 2559 kubelet.go:2448] "Starting kubelet main sync loop" Sep 09 06:38:21.664446 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:21.664418 2559 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 09 06:38:21.666093 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.666072 2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 06:38:21.703014 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:21.702989 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-137.ec2.internal\" not found" Sep 09 06:38:21.765557 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.765500 2559 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-137.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-137.ec2.internal"] Sep 09 06:38:21.765675 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.765592 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 06:38:21.767471 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.767456 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-137.ec2.internal" event="NodeHasSufficientMemory" Sep 09 06:38:21.767558 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.767486 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-137.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 06:38:21.767558 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.767500 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-137.ec2.internal" event="NodeHasSufficientPID" Sep 09 06:38:21.769849 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.769837 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 06:38:21.769992 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.769980 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-137.ec2.internal" Sep 09 06:38:21.770027 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.770009 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 06:38:21.770619 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.770603 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-137.ec2.internal" event="NodeHasSufficientMemory" Sep 09 06:38:21.770713 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.770618 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-137.ec2.internal" event="NodeHasSufficientMemory" Sep 09 06:38:21.770713 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.770644 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-137.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 06:38:21.770713 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.770659 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-137.ec2.internal" event="NodeHasSufficientPID" Sep 09 06:38:21.770713 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.770625 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-137.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 06:38:21.770713 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.770712 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-137.ec2.internal" event="NodeHasSufficientPID" Sep 09 06:38:21.772871 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.772858 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-137.ec2.internal" Sep 09 06:38:21.772918 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.772884 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 06:38:21.773567 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.773552 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-137.ec2.internal" event="NodeHasSufficientMemory" Sep 09 06:38:21.773652 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.773576 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-137.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 06:38:21.773652 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.773586 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-137.ec2.internal" event="NodeHasSufficientPID" Sep 09 06:38:21.794806 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:21.794787 2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-137.ec2.internal\" not found" node="ip-10-0-132-137.ec2.internal" Sep 09 06:38:21.799108 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:21.799094 2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-137.ec2.internal\" not found" node="ip-10-0-132-137.ec2.internal" Sep 09 06:38:21.803530 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:21.803516 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-137.ec2.internal\" not found" Sep 09 06:38:21.827899 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.827879 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/323664165661da7466696aea846c55ee-config\") pod \"kube-apiserver-proxy-ip-10-0-132-137.ec2.internal\" (UID: \"323664165661da7466696aea846c55ee\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-137.ec2.internal" Sep 09 06:38:21.827965 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.827902 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c3546eccb5a037f684d1c790f3c8d333-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-137.ec2.internal\" (UID: \"c3546eccb5a037f684d1c790f3c8d333\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-137.ec2.internal" Sep 09 06:38:21.827965 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.827919 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3546eccb5a037f684d1c790f3c8d333-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-137.ec2.internal\" (UID: \"c3546eccb5a037f684d1c790f3c8d333\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-137.ec2.internal" Sep 09 06:38:21.903973 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:21.903926 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-137.ec2.internal\" not found" Sep 09 06:38:21.928443 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.928422 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3546eccb5a037f684d1c790f3c8d333-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-137.ec2.internal\" (UID: \"c3546eccb5a037f684d1c790f3c8d333\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-137.ec2.internal" Sep 09 06:38:21.928502 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.928449 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/323664165661da7466696aea846c55ee-config\") pod \"kube-apiserver-proxy-ip-10-0-132-137.ec2.internal\" (UID: \"323664165661da7466696aea846c55ee\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-137.ec2.internal" Sep 09 06:38:21.928502 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.928464 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c3546eccb5a037f684d1c790f3c8d333-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-137.ec2.internal\" (UID: \"c3546eccb5a037f684d1c790f3c8d333\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-137.ec2.internal" Sep 09 06:38:21.928604 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.928507 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c3546eccb5a037f684d1c790f3c8d333-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-137.ec2.internal\" (UID: \"c3546eccb5a037f684d1c790f3c8d333\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-137.ec2.internal" Sep 09 06:38:21.928604 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.928523 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3546eccb5a037f684d1c790f3c8d333-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-137.ec2.internal\" (UID: \"c3546eccb5a037f684d1c790f3c8d333\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-137.ec2.internal" Sep 09 06:38:21.928604 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:21.928548 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/323664165661da7466696aea846c55ee-config\") pod \"kube-apiserver-proxy-ip-10-0-132-137.ec2.internal\" (UID: \"323664165661da7466696aea846c55ee\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-137.ec2.internal" Sep 09 06:38:22.004583 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:22.004548 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-137.ec2.internal\" not found" Sep 09 06:38:22.098044 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:22.098018 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-137.ec2.internal" Sep 09 06:38:22.101622 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:22.101601 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-137.ec2.internal" Sep 09 06:38:22.104789 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:22.104771 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-137.ec2.internal\" not found" Sep 09 06:38:23.237231 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:23.237164 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-137.ec2.internal\" not found" Sep 09 06:38:23.337754 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:23.337725 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-137.ec2.internal\" not found" Sep 09 06:38:23.387192 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:23.387170 2559 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 09 06:38:23.387614 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:23.387320 2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 09 06:38:23.438463 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:23.438440 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-137.ec2.internal\" not found" Sep 09 06:38:23.473127 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:23.473098 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-08 06:33:22 +0000 UTC" deadline="2028-06-04 17:56:15.859902088 +0000 UTC" Sep 09 06:38:23.473127 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:23.473121 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15227h17m52.386783519s" Sep 09 06:38:23.482215 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:23.482195 2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 06:38:23.539199 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:23.539149 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-137.ec2.internal\" not found" Sep 09 06:38:23.639686 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:23.639663 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-137.ec2.internal\" not found" Sep 09 06:38:23.740189 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:23.740169 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-137.ec2.internal\" not found" Sep 09 06:38:23.840735 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:23.840708 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-137.ec2.internal\" not found" Sep 09 06:38:23.884660 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:23.883438 2559 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 06:38:23.911188 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:23.911160 2559 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 06:38:23.956179 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:23.956158 2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-137.ec2.internal" Sep 09 06:38:23.966104 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:23.966086 2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 09 06:38:23.967006 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:23.966989 2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-137.ec2.internal" Sep 09 06:38:23.971460 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:23.971431 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3546eccb5a037f684d1c790f3c8d333.slice/crio-f7f3c0811139e6aa70ca92ee0bd768ac43ccc9a7eb81822ba0b85888362bcd48 WatchSource:0}: Error finding container f7f3c0811139e6aa70ca92ee0bd768ac43ccc9a7eb81822ba0b85888362bcd48: Status 404 returned error can't find the container with id f7f3c0811139e6aa70ca92ee0bd768ac43ccc9a7eb81822ba0b85888362bcd48 Sep 09 06:38:23.977726 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:23.977708 2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 09 06:38:23.980256 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:23.980238 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 06:38:23.985562 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:23.985544 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod323664165661da7466696aea846c55ee.slice/crio-56822e890e404637d99f48ba43f5b86333a4d938f9943da1cf2fb23750cf1723 WatchSource:0}: Error finding container 56822e890e404637d99f48ba43f5b86333a4d938f9943da1cf2fb23750cf1723: Status 404 returned error can't find the container with id 56822e890e404637d99f48ba43f5b86333a4d938f9943da1cf2fb23750cf1723 Sep 09 06:38:24.438897 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.438866 2559 apiserver.go:52] "Watching apiserver" Sep 09 06:38:24.445428 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.445409 2559 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 09 06:38:24.448287 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.448262 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-bk8ww","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-137.ec2.internal","openshift-multus/network-metrics-daemon-pkv72","openshift-network-diagnostics/network-check-target-9f6j7","openshift-network-operator/iptables-alerter-5qd9z","openshift-ovn-kubernetes/ovnkube-node-snmsv","kube-system/global-pull-secret-syncer-7rdlw","kube-system/konnectivity-agent-kt6jc","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9dd8h","openshift-cluster-node-tuning-operator/tuned-w776r","openshift-image-registry/node-ca-qr5vp","openshift-multus/multus-additional-cni-plugins-27vp7","openshift-multus/multus-pfbsb","kube-system/kube-apiserver-proxy-ip-10-0-132-137.ec2.internal"] Sep 09 06:38:24.450984 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.450968 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7rdlw" Sep 09 06:38:24.451062 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:24.451035 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7rdlw" podUID="11156d42-6fd0-4887-9f80-6f172dcce6c1" Sep 09 06:38:24.452978 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.452965 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.454944 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.454909 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 09 06:38:24.454944 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.454932 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 09 06:38:24.455129 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.454968 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-vrtpx\"" Sep 09 06:38:24.455129 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.455013 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 09 06:38:24.455215 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.455153 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv72" Sep 09 06:38:24.455215 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.455190 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 09 06:38:24.455295 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:24.455211 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkv72" podUID="336552c8-ca0f-434a-8353-bc61ea118fa5" Sep 09 06:38:24.456878 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.456862 2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 09 06:38:24.457232 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.457215 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f6j7" Sep 09 06:38:24.457302 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:24.457283 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9f6j7" podUID="3355475e-185e-4637-900a-5bf30d29d199" Sep 09 06:38:24.459322 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.459309 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5qd9z" Sep 09 06:38:24.461581 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.461552 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 09 06:38:24.461581 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.461563 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 09 06:38:24.461721 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.461558 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-n2fgd\"" Sep 09 06:38:24.461721 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.461553 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 09 06:38:24.461721 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.461674 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.463797 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.463774 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 09 06:38:24.463878 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.463838 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-kt6jc" Sep 09 06:38:24.464486 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.464252 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 09 06:38:24.464486 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.464264 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 09 06:38:24.464486 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.464291 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-p2cq6\"" Sep 09 06:38:24.464486 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.464272 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 09 06:38:24.464486 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.464354 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 09 06:38:24.464705 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.464632 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 09 06:38:24.466234 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.465919 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 09 06:38:24.466323 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.466304 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-n5pp4\"" Sep 09 06:38:24.466843 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.466814 2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 09 06:38:24.467303 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.467288 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/11156d42-6fd0-4887-9f80-6f172dcce6c1-original-pull-secret\") pod \"global-pull-secret-syncer-7rdlw\" (UID: \"11156d42-6fd0-4887-9f80-6f172dcce6c1\") " pod="kube-system/global-pull-secret-syncer-7rdlw" Sep 09 06:38:24.467383 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.467301 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 09 06:38:24.467383 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.467324 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-multus-cni-dir\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.467383 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.467346 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aeac7db8-f053-4218-9e15-de29565b7826-cni-binary-copy\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.467383 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.467367 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-host-var-lib-cni-multus\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.467564 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.467392 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-host-var-lib-kubelet\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.467564 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.467416 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/11156d42-6fd0-4887-9f80-6f172dcce6c1-kubelet-config\") pod \"global-pull-secret-syncer-7rdlw\" (UID: \"11156d42-6fd0-4887-9f80-6f172dcce6c1\") " pod="kube-system/global-pull-secret-syncer-7rdlw" Sep 09 06:38:24.467564 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.467437 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-cnibin\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.467564 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.467451 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-hostroot\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.467564 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.467475 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-multus-conf-dir\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.467564 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.467540 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/aeac7db8-f053-4218-9e15-de29565b7826-multus-daemon-config\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.468184 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.468046 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-etc-kubernetes\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.468184 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.468169 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhwxb\" (UniqueName: \"kubernetes.io/projected/3355475e-185e-4637-900a-5bf30d29d199-kube-api-access-fhwxb\") pod \"network-check-target-9f6j7\" (UID: \"3355475e-185e-4637-900a-5bf30d29d199\") " pod="openshift-network-diagnostics/network-check-target-9f6j7" Sep 09 06:38:24.468336 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.468238 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bk8ww" Sep 09 06:38:24.468336 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.468249 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/11156d42-6fd0-4887-9f80-6f172dcce6c1-dbus\") pod \"global-pull-secret-syncer-7rdlw\" (UID: \"11156d42-6fd0-4887-9f80-6f172dcce6c1\") " pod="kube-system/global-pull-secret-syncer-7rdlw" Sep 09 06:38:24.468336 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.468289 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-system-cni-dir\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.468472 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.468378 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-host-run-k8s-cni-cncf-io\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.468472 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.468414 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/336552c8-ca0f-434a-8353-bc61ea118fa5-metrics-certs\") pod \"network-metrics-daemon-pkv72\" (UID: \"336552c8-ca0f-434a-8353-bc61ea118fa5\") " pod="openshift-multus/network-metrics-daemon-pkv72" Sep 09 06:38:24.468576 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.468509 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phjvn\" (UniqueName: \"kubernetes.io/projected/336552c8-ca0f-434a-8353-bc61ea118fa5-kube-api-access-phjvn\") pod \"network-metrics-daemon-pkv72\" (UID: \"336552c8-ca0f-434a-8353-bc61ea118fa5\") " pod="openshift-multus/network-metrics-daemon-pkv72" Sep 09 06:38:24.468576 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.468547 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-multus-socket-dir-parent\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.468666 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.468572 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-host-run-multus-certs\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.468666 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.468647 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzzz2\" (UniqueName: \"kubernetes.io/projected/aeac7db8-f053-4218-9e15-de29565b7826-kube-api-access-nzzz2\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.468763 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.468693 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-host-run-netns\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.468763 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.468726 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-host-var-lib-cni-bin\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.469966 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.469946 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-27vp7" Sep 09 06:38:24.470389 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.470362 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-v4fvk\"" Sep 09 06:38:24.470473 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.470410 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 09 06:38:24.470473 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.470446 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 09 06:38:24.471864 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.471834 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2mjcj\"" Sep 09 06:38:24.472010 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.471997 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 09 06:38:24.472227 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.472212 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.472568 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.472554 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 09 06:38:24.474036 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.474015 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 09 06:38:24.474139 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.474047 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 09 06:38:24.474315 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.474297 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hvdr9\"" Sep 09 06:38:24.474579 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.474558 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qr5vp" Sep 09 06:38:24.476179 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.476162 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 09 06:38:24.476588 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.476569 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 09 06:38:24.476667 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.476658 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jswp5\"" Sep 09 06:38:24.476729 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.476663 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 09 06:38:24.476866 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.476850 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9dd8h" Sep 09 06:38:24.478605 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.478588 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 09 06:38:24.478753 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.478740 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 09 06:38:24.478892 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.478876 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 09 06:38:24.478937 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.478886 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-29gqj\"" Sep 09 06:38:24.489928 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.489913 2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hdn2b" Sep 09 06:38:24.499867 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.499850 2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hdn2b" Sep 09 06:38:24.557967 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.557945 2559 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 09 06:38:24.569872 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.569848 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-multus-socket-dir-parent\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.569982 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.569900 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2xcx\" (UniqueName: \"kubernetes.io/projected/9c0c8eea-7803-45cf-a327-2d4c39c42360-kube-api-access-m2xcx\") pod \"iptables-alerter-5qd9z\" (UID: \"9c0c8eea-7803-45cf-a327-2d4c39c42360\") " pod="openshift-network-operator/iptables-alerter-5qd9z" Sep 09 06:38:24.569982 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.569930 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e251251d-5f0e-4372-ada0-e70551d79b78-var-lib-kubelet\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.569982 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.569955 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e2ab2571-e051-4e8e-baf4-ce549dbd71ba-etc-selinux\") pod \"aws-ebs-csi-driver-node-9dd8h\" (UID: \"e2ab2571-e051-4e8e-baf4-ce549dbd71ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9dd8h" Sep 09 06:38:24.570164 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.569976 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-multus-socket-dir-parent\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.570164 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.569979 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-host-run-netns\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.570164 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.570033 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-host-run-netns\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.570164 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.570096 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-host-run-netns\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.570164 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.570121 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-run-systemd\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.570164 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.570145 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3d5681e8-6984-40dc-b918-18483d2bed5c-agent-certs\") pod \"konnectivity-agent-kt6jc\" (UID: \"3d5681e8-6984-40dc-b918-18483d2bed5c\") " pod="kube-system/konnectivity-agent-kt6jc" Sep 09 06:38:24.570370 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.570173 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-hostroot\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.570370 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.570200 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-multus-conf-dir\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.570370 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.570236 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-multus-conf-dir\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.570370 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.570270 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-hostroot\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.570370 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.570284 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9c0c8eea-7803-45cf-a327-2d4c39c42360-host-slash\") pod \"iptables-alerter-5qd9z\" (UID: \"9c0c8eea-7803-45cf-a327-2d4c39c42360\") " pod="openshift-network-operator/iptables-alerter-5qd9z" Sep 09 06:38:24.570370 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.570308 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-host-slash\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.570370 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.570330 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-etc-openvswitch\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.570370 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.570343 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e251251d-5f0e-4372-ada0-e70551d79b78-etc-kubernetes\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.570370 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.570368 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v59lf\" (UniqueName: \"kubernetes.io/projected/e251251d-5f0e-4372-ada0-e70551d79b78-kube-api-access-v59lf\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.570735 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.570392 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e2ab2571-e051-4e8e-baf4-ce549dbd71ba-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9dd8h\" (UID: \"e2ab2571-e051-4e8e-baf4-ce549dbd71ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9dd8h" Sep 09 06:38:24.570735 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.570423 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a101aee5-3bb2-4c35-a963-a498c6c86d91-env-overrides\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.570735 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.570464 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f1a72723-5d93-402c-8ec3-768531d2a883-host\") pod \"node-ca-qr5vp\" (UID: \"f1a72723-5d93-402c-8ec3-768531d2a883\") " pod="openshift-image-registry/node-ca-qr5vp" Sep 09 06:38:24.570735 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.570521 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e2ab2571-e051-4e8e-baf4-ce549dbd71ba-sys-fs\") pod \"aws-ebs-csi-driver-node-9dd8h\" (UID: \"e2ab2571-e051-4e8e-baf4-ce549dbd71ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9dd8h" Sep 09 06:38:24.570735 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.570554 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6k8jg\" (UniqueName: \"kubernetes.io/projected/e2ab2571-e051-4e8e-baf4-ce549dbd71ba-kube-api-access-6k8jg\") pod \"aws-ebs-csi-driver-node-9dd8h\" (UID: \"e2ab2571-e051-4e8e-baf4-ce549dbd71ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9dd8h" Sep 09 06:38:24.570735 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.570582 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a101aee5-3bb2-4c35-a963-a498c6c86d91-ovnkube-config\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.570735 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.570615 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stcrq\" (UniqueName: \"kubernetes.io/projected/a101aee5-3bb2-4c35-a963-a498c6c86d91-kube-api-access-stcrq\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.570735 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.570667 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e251251d-5f0e-4372-ada0-e70551d79b78-etc-sysconfig\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.570735 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.570695 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-host-var-lib-cni-bin\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.570735 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.570712 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-host-var-lib-kubelet\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.570735 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.570728 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/aeac7db8-f053-4218-9e15-de29565b7826-multus-daemon-config\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.571169 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.570748 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-host-cni-bin\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.571169 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.570752 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-host-var-lib-cni-bin\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.571169 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.570765 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-host-var-lib-kubelet\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.571169 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.570784 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e251251d-5f0e-4372-ada0-e70551d79b78-etc-modprobe-d\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.571169 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.570857 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e251251d-5f0e-4372-ada0-e70551d79b78-etc-systemd\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.571169 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.570896 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e251251d-5f0e-4372-ada0-e70551d79b78-etc-tuned\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.571169 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.570913 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/13c06be2-78fb-4ceb-83ee-8cc6a545b395-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-27vp7\" (UID: \"13c06be2-78fb-4ceb-83ee-8cc6a545b395\") " pod="openshift-multus/multus-additional-cni-plugins-27vp7" Sep 09 06:38:24.571169 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.570930 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-cnibin\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.571169 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.570952 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-node-log\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.571169 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.570981 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-host-cni-netd\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.571169 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.570997 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-cnibin\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.571169 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.571006 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a101aee5-3bb2-4c35-a963-a498c6c86d91-ovn-node-metrics-cert\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.571169 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.571032 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0f25c33f-2f5d-4645-990b-b9d2f201e1fc-hosts-file\") pod \"node-resolver-bk8ww\" (UID: \"0f25c33f-2f5d-4645-990b-b9d2f201e1fc\") " pod="openshift-dns/node-resolver-bk8ww" Sep 09 06:38:24.571169 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.571067 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-host-run-k8s-cni-cncf-io\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.571169 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.571086 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-var-lib-openvswitch\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.571169 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.571110 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-host-run-ovn-kubernetes\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.571169 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.571133 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.571998 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.571131 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-host-run-k8s-cni-cncf-io\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.571998 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.571161 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e251251d-5f0e-4372-ada0-e70551d79b78-tmp\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.571998 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.571184 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e2ab2571-e051-4e8e-baf4-ce549dbd71ba-socket-dir\") pod \"aws-ebs-csi-driver-node-9dd8h\" (UID: \"e2ab2571-e051-4e8e-baf4-ce549dbd71ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9dd8h" Sep 09 06:38:24.571998 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.571206 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e2ab2571-e051-4e8e-baf4-ce549dbd71ba-registration-dir\") pod \"aws-ebs-csi-driver-node-9dd8h\" (UID: \"e2ab2571-e051-4e8e-baf4-ce549dbd71ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9dd8h" Sep 09 06:38:24.571998 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.571231 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-host-run-multus-certs\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.571998 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.571247 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzzz2\" (UniqueName: \"kubernetes.io/projected/aeac7db8-f053-4218-9e15-de29565b7826-kube-api-access-nzzz2\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.571998 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.571268 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e2ab2571-e051-4e8e-baf4-ce549dbd71ba-device-dir\") pod \"aws-ebs-csi-driver-node-9dd8h\" (UID: \"e2ab2571-e051-4e8e-baf4-ce549dbd71ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9dd8h" Sep 09 06:38:24.571998 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.571297 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/11156d42-6fd0-4887-9f80-6f172dcce6c1-original-pull-secret\") pod \"global-pull-secret-syncer-7rdlw\" (UID: \"11156d42-6fd0-4887-9f80-6f172dcce6c1\") " pod="kube-system/global-pull-secret-syncer-7rdlw" Sep 09 06:38:24.571998 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.571306 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-host-run-multus-certs\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.571998 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.571332 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-host-var-lib-cni-multus\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.571998 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.571369 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-host-var-lib-cni-multus\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.571998 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.571380 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-host-kubelet\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.571998 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.571407 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-run-openvswitch\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.571998 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.571460 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-run-ovn\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.571998 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.571504 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/13c06be2-78fb-4ceb-83ee-8cc6a545b395-system-cni-dir\") pod \"multus-additional-cni-plugins-27vp7\" (UID: \"13c06be2-78fb-4ceb-83ee-8cc6a545b395\") " pod="openshift-multus/multus-additional-cni-plugins-27vp7" Sep 09 06:38:24.571998 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.571539 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nc42\" (UniqueName: \"kubernetes.io/projected/13c06be2-78fb-4ceb-83ee-8cc6a545b395-kube-api-access-7nc42\") pod \"multus-additional-cni-plugins-27vp7\" (UID: \"13c06be2-78fb-4ceb-83ee-8cc6a545b395\") " pod="openshift-multus/multus-additional-cni-plugins-27vp7" Sep 09 06:38:24.571998 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.571579 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e251251d-5f0e-4372-ada0-e70551d79b78-etc-sysctl-d\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.572719 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.571614 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/11156d42-6fd0-4887-9f80-6f172dcce6c1-kubelet-config\") pod \"global-pull-secret-syncer-7rdlw\" (UID: \"11156d42-6fd0-4887-9f80-6f172dcce6c1\") " pod="kube-system/global-pull-secret-syncer-7rdlw" Sep 09 06:38:24.572719 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.571646 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e251251d-5f0e-4372-ada0-e70551d79b78-etc-sysctl-conf\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.572719 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.571667 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e251251d-5f0e-4372-ada0-e70551d79b78-run\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.572719 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.571691 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e251251d-5f0e-4372-ada0-e70551d79b78-lib-modules\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.572719 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.571699 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/11156d42-6fd0-4887-9f80-6f172dcce6c1-kubelet-config\") pod \"global-pull-secret-syncer-7rdlw\" (UID: \"11156d42-6fd0-4887-9f80-6f172dcce6c1\") " pod="kube-system/global-pull-secret-syncer-7rdlw" Sep 09 06:38:24.572719 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.571713 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p4cr\" (UniqueName: \"kubernetes.io/projected/f1a72723-5d93-402c-8ec3-768531d2a883-kube-api-access-2p4cr\") pod \"node-ca-qr5vp\" (UID: \"f1a72723-5d93-402c-8ec3-768531d2a883\") " pod="openshift-image-registry/node-ca-qr5vp" Sep 09 06:38:24.572719 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.571754 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/336552c8-ca0f-434a-8353-bc61ea118fa5-metrics-certs\") pod \"network-metrics-daemon-pkv72\" (UID: \"336552c8-ca0f-434a-8353-bc61ea118fa5\") " pod="openshift-multus/network-metrics-daemon-pkv72" Sep 09 06:38:24.572719 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.571775 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/13c06be2-78fb-4ceb-83ee-8cc6a545b395-cnibin\") pod \"multus-additional-cni-plugins-27vp7\" (UID: \"13c06be2-78fb-4ceb-83ee-8cc6a545b395\") " pod="openshift-multus/multus-additional-cni-plugins-27vp7" Sep 09 06:38:24.572719 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.571795 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e251251d-5f0e-4372-ada0-e70551d79b78-host\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.572719 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.571823 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9c0c8eea-7803-45cf-a327-2d4c39c42360-iptables-alerter-script\") pod \"iptables-alerter-5qd9z\" (UID: \"9c0c8eea-7803-45cf-a327-2d4c39c42360\") " pod="openshift-network-operator/iptables-alerter-5qd9z" Sep 09 06:38:24.572719 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:24.571860 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 06:38:24.572719 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.571883 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-systemd-units\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.572719 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.571908 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f25c33f-2f5d-4645-990b-b9d2f201e1fc-tmp-dir\") pod \"node-resolver-bk8ww\" (UID: \"0f25c33f-2f5d-4645-990b-b9d2f201e1fc\") " pod="openshift-dns/node-resolver-bk8ww" Sep 09 06:38:24.572719 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:24.571946 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/336552c8-ca0f-434a-8353-bc61ea118fa5-metrics-certs podName:336552c8-ca0f-434a-8353-bc61ea118fa5 nodeName:}" failed. No retries permitted until 2026-09-09 06:38:25.07191307 +0000 UTC m=+2.968330476 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/336552c8-ca0f-434a-8353-bc61ea118fa5-metrics-certs") pod "network-metrics-daemon-pkv72" (UID: "336552c8-ca0f-434a-8353-bc61ea118fa5") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 06:38:24.572719 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:24.571862 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 06:38:24.572719 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.571981 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/13c06be2-78fb-4ceb-83ee-8cc6a545b395-tuning-conf-dir\") pod \"multus-additional-cni-plugins-27vp7\" (UID: \"13c06be2-78fb-4ceb-83ee-8cc6a545b395\") " pod="openshift-multus/multus-additional-cni-plugins-27vp7" Sep 09 06:38:24.572719 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.572275 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e251251d-5f0e-4372-ada0-e70551d79b78-sys\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.573500 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:24.572286 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11156d42-6fd0-4887-9f80-6f172dcce6c1-original-pull-secret podName:11156d42-6fd0-4887-9f80-6f172dcce6c1 nodeName:}" failed. No retries permitted until 2026-09-09 06:38:25.072258046 +0000 UTC m=+2.968675460 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/11156d42-6fd0-4887-9f80-6f172dcce6c1-original-pull-secret") pod "global-pull-secret-syncer-7rdlw" (UID: "11156d42-6fd0-4887-9f80-6f172dcce6c1") : object "kube-system"/"original-pull-secret" not registered Sep 09 06:38:24.573500 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.572348 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f1a72723-5d93-402c-8ec3-768531d2a883-serviceca\") pod \"node-ca-qr5vp\" (UID: \"f1a72723-5d93-402c-8ec3-768531d2a883\") " pod="openshift-image-registry/node-ca-qr5vp" Sep 09 06:38:24.573500 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.572385 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-multus-cni-dir\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.573500 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.572422 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aeac7db8-f053-4218-9e15-de29565b7826-cni-binary-copy\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.573500 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.572458 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhwxb\" (UniqueName: \"kubernetes.io/projected/3355475e-185e-4637-900a-5bf30d29d199-kube-api-access-fhwxb\") pod \"network-check-target-9f6j7\" (UID: \"3355475e-185e-4637-900a-5bf30d29d199\") " pod="openshift-network-diagnostics/network-check-target-9f6j7" Sep 09 06:38:24.573500 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.572457 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/aeac7db8-f053-4218-9e15-de29565b7826-multus-daemon-config\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.573500 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.572505 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-log-socket\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.573500 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.572647 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gghw\" (UniqueName: \"kubernetes.io/projected/0f25c33f-2f5d-4645-990b-b9d2f201e1fc-kube-api-access-5gghw\") pod \"node-resolver-bk8ww\" (UID: \"0f25c33f-2f5d-4645-990b-b9d2f201e1fc\") " pod="openshift-dns/node-resolver-bk8ww" Sep 09 06:38:24.573500 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.572652 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-multus-cni-dir\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.573500 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.572724 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3d5681e8-6984-40dc-b918-18483d2bed5c-konnectivity-ca\") pod \"konnectivity-agent-kt6jc\" (UID: \"3d5681e8-6984-40dc-b918-18483d2bed5c\") " pod="kube-system/konnectivity-agent-kt6jc" Sep 09 06:38:24.573500 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.572775 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-etc-kubernetes\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.573500 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.572836 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a101aee5-3bb2-4c35-a963-a498c6c86d91-ovnkube-script-lib\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.573500 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.572911 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/13c06be2-78fb-4ceb-83ee-8cc6a545b395-cni-binary-copy\") pod \"multus-additional-cni-plugins-27vp7\" (UID: \"13c06be2-78fb-4ceb-83ee-8cc6a545b395\") " pod="openshift-multus/multus-additional-cni-plugins-27vp7" Sep 09 06:38:24.573500 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.572985 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/11156d42-6fd0-4887-9f80-6f172dcce6c1-dbus\") pod \"global-pull-secret-syncer-7rdlw\" (UID: \"11156d42-6fd0-4887-9f80-6f172dcce6c1\") " pod="kube-system/global-pull-secret-syncer-7rdlw" Sep 09 06:38:24.573500 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.573045 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-system-cni-dir\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.573500 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.573094 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phjvn\" (UniqueName: \"kubernetes.io/projected/336552c8-ca0f-434a-8353-bc61ea118fa5-kube-api-access-phjvn\") pod \"network-metrics-daemon-pkv72\" (UID: \"336552c8-ca0f-434a-8353-bc61ea118fa5\") " pod="openshift-multus/network-metrics-daemon-pkv72" Sep 09 06:38:24.574230 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.573139 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-etc-kubernetes\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.574230 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.573230 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aeac7db8-f053-4218-9e15-de29565b7826-system-cni-dir\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.574230 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.573299 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/11156d42-6fd0-4887-9f80-6f172dcce6c1-dbus\") pod \"global-pull-secret-syncer-7rdlw\" (UID: \"11156d42-6fd0-4887-9f80-6f172dcce6c1\") " pod="kube-system/global-pull-secret-syncer-7rdlw" Sep 09 06:38:24.574230 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.573340 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/13c06be2-78fb-4ceb-83ee-8cc6a545b395-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-27vp7\" (UID: \"13c06be2-78fb-4ceb-83ee-8cc6a545b395\") " pod="openshift-multus/multus-additional-cni-plugins-27vp7" Sep 09 06:38:24.574230 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.573759 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aeac7db8-f053-4218-9e15-de29565b7826-cni-binary-copy\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.578852 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.578828 2559 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 09 06:38:24.580105 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:24.580087 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 06:38:24.580211 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:24.580108 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 06:38:24.580211 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:24.580121 2559 projected.go:196] Error preparing data for projected volume kube-api-access-fhwxb for pod openshift-network-diagnostics/network-check-target-9f6j7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 06:38:24.580211 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:24.580178 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3355475e-185e-4637-900a-5bf30d29d199-kube-api-access-fhwxb podName:3355475e-185e-4637-900a-5bf30d29d199 nodeName:}" failed. No retries permitted until 2026-09-09 06:38:25.08016235 +0000 UTC m=+2.976579763 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-fhwxb" (UniqueName: "kubernetes.io/projected/3355475e-185e-4637-900a-5bf30d29d199-kube-api-access-fhwxb") pod "network-check-target-9f6j7" (UID: "3355475e-185e-4637-900a-5bf30d29d199") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 06:38:24.582824 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.582798 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzzz2\" (UniqueName: \"kubernetes.io/projected/aeac7db8-f053-4218-9e15-de29565b7826-kube-api-access-nzzz2\") pod \"multus-pfbsb\" (UID: \"aeac7db8-f053-4218-9e15-de29565b7826\") " pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.582983 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.582962 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phjvn\" (UniqueName: \"kubernetes.io/projected/336552c8-ca0f-434a-8353-bc61ea118fa5-kube-api-access-phjvn\") pod \"network-metrics-daemon-pkv72\" (UID: \"336552c8-ca0f-434a-8353-bc61ea118fa5\") " pod="openshift-multus/network-metrics-daemon-pkv72" Sep 09 06:38:24.674435 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.674405 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-log-socket\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.674590 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.674441 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gghw\" (UniqueName: \"kubernetes.io/projected/0f25c33f-2f5d-4645-990b-b9d2f201e1fc-kube-api-access-5gghw\") pod \"node-resolver-bk8ww\" (UID: \"0f25c33f-2f5d-4645-990b-b9d2f201e1fc\") " pod="openshift-dns/node-resolver-bk8ww" Sep 09 06:38:24.674590 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.674466 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3d5681e8-6984-40dc-b918-18483d2bed5c-konnectivity-ca\") pod \"konnectivity-agent-kt6jc\" (UID: \"3d5681e8-6984-40dc-b918-18483d2bed5c\") " pod="kube-system/konnectivity-agent-kt6jc" Sep 09 06:38:24.674590 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.674532 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-log-socket\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.674590 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.674581 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a101aee5-3bb2-4c35-a963-a498c6c86d91-ovnkube-script-lib\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.674800 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.674608 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/13c06be2-78fb-4ceb-83ee-8cc6a545b395-cni-binary-copy\") pod \"multus-additional-cni-plugins-27vp7\" (UID: \"13c06be2-78fb-4ceb-83ee-8cc6a545b395\") " pod="openshift-multus/multus-additional-cni-plugins-27vp7" Sep 09 06:38:24.674800 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.674638 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/13c06be2-78fb-4ceb-83ee-8cc6a545b395-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-27vp7\" (UID: \"13c06be2-78fb-4ceb-83ee-8cc6a545b395\") " pod="openshift-multus/multus-additional-cni-plugins-27vp7" Sep 09 06:38:24.674800 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.674665 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2xcx\" (UniqueName: \"kubernetes.io/projected/9c0c8eea-7803-45cf-a327-2d4c39c42360-kube-api-access-m2xcx\") pod \"iptables-alerter-5qd9z\" (UID: \"9c0c8eea-7803-45cf-a327-2d4c39c42360\") " pod="openshift-network-operator/iptables-alerter-5qd9z" Sep 09 06:38:24.674800 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.674689 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e251251d-5f0e-4372-ada0-e70551d79b78-var-lib-kubelet\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.674800 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.674724 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e2ab2571-e051-4e8e-baf4-ce549dbd71ba-etc-selinux\") pod \"aws-ebs-csi-driver-node-9dd8h\" (UID: \"e2ab2571-e051-4e8e-baf4-ce549dbd71ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9dd8h" Sep 09 06:38:24.674800 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.674749 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-host-run-netns\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.674800 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.674779 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-run-systemd\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.676383 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.674803 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3d5681e8-6984-40dc-b918-18483d2bed5c-agent-certs\") pod \"konnectivity-agent-kt6jc\" (UID: \"3d5681e8-6984-40dc-b918-18483d2bed5c\") " pod="kube-system/konnectivity-agent-kt6jc" Sep 09 06:38:24.676383 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.674830 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9c0c8eea-7803-45cf-a327-2d4c39c42360-host-slash\") pod \"iptables-alerter-5qd9z\" (UID: \"9c0c8eea-7803-45cf-a327-2d4c39c42360\") " pod="openshift-network-operator/iptables-alerter-5qd9z" Sep 09 06:38:24.676383 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.674839 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e251251d-5f0e-4372-ada0-e70551d79b78-var-lib-kubelet\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.676383 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.674855 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-host-slash\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.676383 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.674855 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-host-run-netns\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.676383 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.674884 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-etc-openvswitch\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.676383 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.674908 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e251251d-5f0e-4372-ada0-e70551d79b78-etc-kubernetes\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.676383 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.674936 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v59lf\" (UniqueName: \"kubernetes.io/projected/e251251d-5f0e-4372-ada0-e70551d79b78-kube-api-access-v59lf\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.676383 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.674982 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e2ab2571-e051-4e8e-baf4-ce549dbd71ba-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9dd8h\" (UID: \"e2ab2571-e051-4e8e-baf4-ce549dbd71ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9dd8h" Sep 09 06:38:24.676383 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675006 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a101aee5-3bb2-4c35-a963-a498c6c86d91-env-overrides\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.676383 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675029 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f1a72723-5d93-402c-8ec3-768531d2a883-host\") pod \"node-ca-qr5vp\" (UID: \"f1a72723-5d93-402c-8ec3-768531d2a883\") " pod="openshift-image-registry/node-ca-qr5vp" Sep 09 06:38:24.676383 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675075 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e2ab2571-e051-4e8e-baf4-ce549dbd71ba-sys-fs\") pod \"aws-ebs-csi-driver-node-9dd8h\" (UID: \"e2ab2571-e051-4e8e-baf4-ce549dbd71ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9dd8h" Sep 09 06:38:24.676383 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675101 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k8jg\" (UniqueName: \"kubernetes.io/projected/e2ab2571-e051-4e8e-baf4-ce549dbd71ba-kube-api-access-6k8jg\") pod \"aws-ebs-csi-driver-node-9dd8h\" (UID: \"e2ab2571-e051-4e8e-baf4-ce549dbd71ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9dd8h" Sep 09 06:38:24.676383 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675132 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3d5681e8-6984-40dc-b918-18483d2bed5c-konnectivity-ca\") pod \"konnectivity-agent-kt6jc\" (UID: \"3d5681e8-6984-40dc-b918-18483d2bed5c\") " pod="kube-system/konnectivity-agent-kt6jc" Sep 09 06:38:24.676383 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675145 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e251251d-5f0e-4372-ada0-e70551d79b78-etc-kubernetes\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.676383 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675148 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-run-systemd\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.676383 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675150 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a101aee5-3bb2-4c35-a963-a498c6c86d91-ovnkube-config\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.677273 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675200 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stcrq\" (UniqueName: \"kubernetes.io/projected/a101aee5-3bb2-4c35-a963-a498c6c86d91-kube-api-access-stcrq\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.677273 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675217 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-etc-openvswitch\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.677273 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675227 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e251251d-5f0e-4372-ada0-e70551d79b78-etc-sysconfig\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.677273 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.674904 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e2ab2571-e051-4e8e-baf4-ce549dbd71ba-etc-selinux\") pod \"aws-ebs-csi-driver-node-9dd8h\" (UID: \"e2ab2571-e051-4e8e-baf4-ce549dbd71ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9dd8h" Sep 09 06:38:24.677273 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675244 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a101aee5-3bb2-4c35-a963-a498c6c86d91-ovnkube-script-lib\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.677273 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675257 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-host-cni-bin\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.677273 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675280 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e251251d-5f0e-4372-ada0-e70551d79b78-etc-modprobe-d\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.677273 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675306 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e251251d-5f0e-4372-ada0-e70551d79b78-etc-systemd\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.677273 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675308 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e2ab2571-e051-4e8e-baf4-ce549dbd71ba-sys-fs\") pod \"aws-ebs-csi-driver-node-9dd8h\" (UID: \"e2ab2571-e051-4e8e-baf4-ce549dbd71ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9dd8h" Sep 09 06:38:24.677273 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675338 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e251251d-5f0e-4372-ada0-e70551d79b78-etc-tuned\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.677273 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675365 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/13c06be2-78fb-4ceb-83ee-8cc6a545b395-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-27vp7\" (UID: \"13c06be2-78fb-4ceb-83ee-8cc6a545b395\") " pod="openshift-multus/multus-additional-cni-plugins-27vp7" Sep 09 06:38:24.677273 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675392 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-node-log\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.677273 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675419 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-host-cni-netd\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.677273 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675445 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a101aee5-3bb2-4c35-a963-a498c6c86d91-ovn-node-metrics-cert\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.677273 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675447 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/13c06be2-78fb-4ceb-83ee-8cc6a545b395-cni-binary-copy\") pod \"multus-additional-cni-plugins-27vp7\" (UID: \"13c06be2-78fb-4ceb-83ee-8cc6a545b395\") " pod="openshift-multus/multus-additional-cni-plugins-27vp7" Sep 09 06:38:24.677273 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675448 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/13c06be2-78fb-4ceb-83ee-8cc6a545b395-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-27vp7\" (UID: \"13c06be2-78fb-4ceb-83ee-8cc6a545b395\") " pod="openshift-multus/multus-additional-cni-plugins-27vp7" Sep 09 06:38:24.677273 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675506 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e2ab2571-e051-4e8e-baf4-ce549dbd71ba-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9dd8h\" (UID: \"e2ab2571-e051-4e8e-baf4-ce549dbd71ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9dd8h" Sep 09 06:38:24.677874 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675470 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0f25c33f-2f5d-4645-990b-b9d2f201e1fc-hosts-file\") pod \"node-resolver-bk8ww\" (UID: \"0f25c33f-2f5d-4645-990b-b9d2f201e1fc\") " pod="openshift-dns/node-resolver-bk8ww" Sep 09 06:38:24.677874 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675531 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f1a72723-5d93-402c-8ec3-768531d2a883-host\") pod \"node-ca-qr5vp\" (UID: \"f1a72723-5d93-402c-8ec3-768531d2a883\") " pod="openshift-image-registry/node-ca-qr5vp" Sep 09 06:38:24.677874 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675544 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0f25c33f-2f5d-4645-990b-b9d2f201e1fc-hosts-file\") pod \"node-resolver-bk8ww\" (UID: \"0f25c33f-2f5d-4645-990b-b9d2f201e1fc\") " pod="openshift-dns/node-resolver-bk8ww" Sep 09 06:38:24.677874 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675560 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-var-lib-openvswitch\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.677874 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675578 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e251251d-5f0e-4372-ada0-e70551d79b78-etc-systemd\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.677874 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675589 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-host-run-ovn-kubernetes\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.677874 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675615 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-node-log\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.677874 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675618 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.677874 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675666 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a101aee5-3bb2-4c35-a963-a498c6c86d91-ovnkube-config\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.677874 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675673 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e251251d-5f0e-4372-ada0-e70551d79b78-tmp\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.677874 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675615 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-host-cni-netd\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.677874 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675113 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9c0c8eea-7803-45cf-a327-2d4c39c42360-host-slash\") pod \"iptables-alerter-5qd9z\" (UID: \"9c0c8eea-7803-45cf-a327-2d4c39c42360\") " pod="openshift-network-operator/iptables-alerter-5qd9z" Sep 09 06:38:24.677874 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675509 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e251251d-5f0e-4372-ada0-e70551d79b78-etc-sysconfig\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.677874 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675698 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e2ab2571-e051-4e8e-baf4-ce549dbd71ba-socket-dir\") pod \"aws-ebs-csi-driver-node-9dd8h\" (UID: \"e2ab2571-e051-4e8e-baf4-ce549dbd71ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9dd8h" Sep 09 06:38:24.677874 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675742 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e2ab2571-e051-4e8e-baf4-ce549dbd71ba-registration-dir\") pod \"aws-ebs-csi-driver-node-9dd8h\" (UID: \"e2ab2571-e051-4e8e-baf4-ce549dbd71ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9dd8h" Sep 09 06:38:24.677874 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675751 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-host-slash\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.677874 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675749 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-var-lib-openvswitch\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.678640 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675768 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e2ab2571-e051-4e8e-baf4-ce549dbd71ba-device-dir\") pod \"aws-ebs-csi-driver-node-9dd8h\" (UID: \"e2ab2571-e051-4e8e-baf4-ce549dbd71ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9dd8h" Sep 09 06:38:24.678640 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675800 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-host-cni-bin\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.678640 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675829 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.678640 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675836 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-host-kubelet\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.678640 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675804 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-host-kubelet\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.678640 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675849 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e2ab2571-e051-4e8e-baf4-ce549dbd71ba-socket-dir\") pod \"aws-ebs-csi-driver-node-9dd8h\" (UID: \"e2ab2571-e051-4e8e-baf4-ce549dbd71ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9dd8h" Sep 09 06:38:24.678640 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675870 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-host-run-ovn-kubernetes\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.678640 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675873 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-run-openvswitch\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.678640 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675904 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-run-openvswitch\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.678640 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675910 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-run-ovn\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.678640 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675939 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/13c06be2-78fb-4ceb-83ee-8cc6a545b395-system-cni-dir\") pod \"multus-additional-cni-plugins-27vp7\" (UID: \"13c06be2-78fb-4ceb-83ee-8cc6a545b395\") " pod="openshift-multus/multus-additional-cni-plugins-27vp7" Sep 09 06:38:24.678640 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675957 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e2ab2571-e051-4e8e-baf4-ce549dbd71ba-registration-dir\") pod \"aws-ebs-csi-driver-node-9dd8h\" (UID: \"e2ab2571-e051-4e8e-baf4-ce549dbd71ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9dd8h" Sep 09 06:38:24.678640 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675957 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e251251d-5f0e-4372-ada0-e70551d79b78-etc-modprobe-d\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.678640 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675966 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nc42\" (UniqueName: \"kubernetes.io/projected/13c06be2-78fb-4ceb-83ee-8cc6a545b395-kube-api-access-7nc42\") pod \"multus-additional-cni-plugins-27vp7\" (UID: \"13c06be2-78fb-4ceb-83ee-8cc6a545b395\") " pod="openshift-multus/multus-additional-cni-plugins-27vp7" Sep 09 06:38:24.678640 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675995 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e251251d-5f0e-4372-ada0-e70551d79b78-etc-sysctl-d\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.678640 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.676000 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-run-ovn\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.678640 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.676022 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e251251d-5f0e-4372-ada0-e70551d79b78-etc-sysctl-conf\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.679470 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.676038 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/13c06be2-78fb-4ceb-83ee-8cc6a545b395-system-cni-dir\") pod \"multus-additional-cni-plugins-27vp7\" (UID: \"13c06be2-78fb-4ceb-83ee-8cc6a545b395\") " pod="openshift-multus/multus-additional-cni-plugins-27vp7" Sep 09 06:38:24.679470 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.676042 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/13c06be2-78fb-4ceb-83ee-8cc6a545b395-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-27vp7\" (UID: \"13c06be2-78fb-4ceb-83ee-8cc6a545b395\") " pod="openshift-multus/multus-additional-cni-plugins-27vp7" Sep 09 06:38:24.679470 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.675997 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e2ab2571-e051-4e8e-baf4-ce549dbd71ba-device-dir\") pod \"aws-ebs-csi-driver-node-9dd8h\" (UID: \"e2ab2571-e051-4e8e-baf4-ce549dbd71ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9dd8h" Sep 09 06:38:24.679470 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.676071 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e251251d-5f0e-4372-ada0-e70551d79b78-run\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.679470 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.676121 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e251251d-5f0e-4372-ada0-e70551d79b78-run\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.679470 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.676176 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e251251d-5f0e-4372-ada0-e70551d79b78-etc-sysctl-d\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.679470 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.676200 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e251251d-5f0e-4372-ada0-e70551d79b78-etc-sysctl-conf\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.679470 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.676302 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e251251d-5f0e-4372-ada0-e70551d79b78-lib-modules\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.679470 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.676330 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2p4cr\" (UniqueName: \"kubernetes.io/projected/f1a72723-5d93-402c-8ec3-768531d2a883-kube-api-access-2p4cr\") pod \"node-ca-qr5vp\" (UID: \"f1a72723-5d93-402c-8ec3-768531d2a883\") " pod="openshift-image-registry/node-ca-qr5vp" Sep 09 06:38:24.679470 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.676365 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/13c06be2-78fb-4ceb-83ee-8cc6a545b395-cnibin\") pod \"multus-additional-cni-plugins-27vp7\" (UID: \"13c06be2-78fb-4ceb-83ee-8cc6a545b395\") " pod="openshift-multus/multus-additional-cni-plugins-27vp7" Sep 09 06:38:24.679470 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.676389 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e251251d-5f0e-4372-ada0-e70551d79b78-host\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.679470 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.676393 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e251251d-5f0e-4372-ada0-e70551d79b78-lib-modules\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.679470 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.676409 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9c0c8eea-7803-45cf-a327-2d4c39c42360-iptables-alerter-script\") pod \"iptables-alerter-5qd9z\" (UID: \"9c0c8eea-7803-45cf-a327-2d4c39c42360\") " pod="openshift-network-operator/iptables-alerter-5qd9z" Sep 09 06:38:24.679470 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.676444 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-systemd-units\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.679470 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.676472 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f25c33f-2f5d-4645-990b-b9d2f201e1fc-tmp-dir\") pod \"node-resolver-bk8ww\" (UID: \"0f25c33f-2f5d-4645-990b-b9d2f201e1fc\") " pod="openshift-dns/node-resolver-bk8ww" Sep 09 06:38:24.679470 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.676494 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/13c06be2-78fb-4ceb-83ee-8cc6a545b395-tuning-conf-dir\") pod \"multus-additional-cni-plugins-27vp7\" (UID: \"13c06be2-78fb-4ceb-83ee-8cc6a545b395\") " pod="openshift-multus/multus-additional-cni-plugins-27vp7" Sep 09 06:38:24.679470 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.676515 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e251251d-5f0e-4372-ada0-e70551d79b78-sys\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.680018 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.676533 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f1a72723-5d93-402c-8ec3-768531d2a883-serviceca\") pod \"node-ca-qr5vp\" (UID: \"f1a72723-5d93-402c-8ec3-768531d2a883\") " pod="openshift-image-registry/node-ca-qr5vp" Sep 09 06:38:24.680018 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.676446 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/13c06be2-78fb-4ceb-83ee-8cc6a545b395-cnibin\") pod \"multus-additional-cni-plugins-27vp7\" (UID: \"13c06be2-78fb-4ceb-83ee-8cc6a545b395\") " pod="openshift-multus/multus-additional-cni-plugins-27vp7" Sep 09 06:38:24.680018 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.676694 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e251251d-5f0e-4372-ada0-e70551d79b78-host\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.680018 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.676945 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f1a72723-5d93-402c-8ec3-768531d2a883-serviceca\") pod \"node-ca-qr5vp\" (UID: \"f1a72723-5d93-402c-8ec3-768531d2a883\") " pod="openshift-image-registry/node-ca-qr5vp" Sep 09 06:38:24.680018 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.677006 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e251251d-5f0e-4372-ada0-e70551d79b78-sys\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.680018 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.677023 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/13c06be2-78fb-4ceb-83ee-8cc6a545b395-tuning-conf-dir\") pod \"multus-additional-cni-plugins-27vp7\" (UID: \"13c06be2-78fb-4ceb-83ee-8cc6a545b395\") " pod="openshift-multus/multus-additional-cni-plugins-27vp7" Sep 09 06:38:24.680018 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.677068 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a101aee5-3bb2-4c35-a963-a498c6c86d91-systemd-units\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.680018 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.677259 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9c0c8eea-7803-45cf-a327-2d4c39c42360-iptables-alerter-script\") pod \"iptables-alerter-5qd9z\" (UID: \"9c0c8eea-7803-45cf-a327-2d4c39c42360\") " pod="openshift-network-operator/iptables-alerter-5qd9z" Sep 09 06:38:24.680018 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.677302 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f25c33f-2f5d-4645-990b-b9d2f201e1fc-tmp-dir\") pod \"node-resolver-bk8ww\" (UID: \"0f25c33f-2f5d-4645-990b-b9d2f201e1fc\") " pod="openshift-dns/node-resolver-bk8ww" Sep 09 06:38:24.680018 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.678481 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3d5681e8-6984-40dc-b918-18483d2bed5c-agent-certs\") pod \"konnectivity-agent-kt6jc\" (UID: \"3d5681e8-6984-40dc-b918-18483d2bed5c\") " pod="kube-system/konnectivity-agent-kt6jc" Sep 09 06:38:24.680018 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.679035 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e251251d-5f0e-4372-ada0-e70551d79b78-tmp\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.680018 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.679142 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a101aee5-3bb2-4c35-a963-a498c6c86d91-env-overrides\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.680018 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.679385 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a101aee5-3bb2-4c35-a963-a498c6c86d91-ovn-node-metrics-cert\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.680018 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.679787 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e251251d-5f0e-4372-ada0-e70551d79b78-etc-tuned\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.683915 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.683889 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gghw\" (UniqueName: \"kubernetes.io/projected/0f25c33f-2f5d-4645-990b-b9d2f201e1fc-kube-api-access-5gghw\") pod \"node-resolver-bk8ww\" (UID: \"0f25c33f-2f5d-4645-990b-b9d2f201e1fc\") " pod="openshift-dns/node-resolver-bk8ww" Sep 09 06:38:24.683993 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.683916 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6k8jg\" (UniqueName: \"kubernetes.io/projected/e2ab2571-e051-4e8e-baf4-ce549dbd71ba-kube-api-access-6k8jg\") pod \"aws-ebs-csi-driver-node-9dd8h\" (UID: \"e2ab2571-e051-4e8e-baf4-ce549dbd71ba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9dd8h" Sep 09 06:38:24.685244 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.685202 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p4cr\" (UniqueName: \"kubernetes.io/projected/f1a72723-5d93-402c-8ec3-768531d2a883-kube-api-access-2p4cr\") pod \"node-ca-qr5vp\" (UID: \"f1a72723-5d93-402c-8ec3-768531d2a883\") " pod="openshift-image-registry/node-ca-qr5vp" Sep 09 06:38:24.686134 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.686112 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v59lf\" (UniqueName: \"kubernetes.io/projected/e251251d-5f0e-4372-ada0-e70551d79b78-kube-api-access-v59lf\") pod \"tuned-w776r\" (UID: \"e251251d-5f0e-4372-ada0-e70551d79b78\") " pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.686781 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.686364 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nc42\" (UniqueName: \"kubernetes.io/projected/13c06be2-78fb-4ceb-83ee-8cc6a545b395-kube-api-access-7nc42\") pod \"multus-additional-cni-plugins-27vp7\" (UID: \"13c06be2-78fb-4ceb-83ee-8cc6a545b395\") " pod="openshift-multus/multus-additional-cni-plugins-27vp7" Sep 09 06:38:24.686781 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.686385 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2xcx\" (UniqueName: \"kubernetes.io/projected/9c0c8eea-7803-45cf-a327-2d4c39c42360-kube-api-access-m2xcx\") pod \"iptables-alerter-5qd9z\" (UID: \"9c0c8eea-7803-45cf-a327-2d4c39c42360\") " pod="openshift-network-operator/iptables-alerter-5qd9z" Sep 09 06:38:24.687450 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.687428 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stcrq\" (UniqueName: \"kubernetes.io/projected/a101aee5-3bb2-4c35-a963-a498c6c86d91-kube-api-access-stcrq\") pod \"ovnkube-node-snmsv\" (UID: \"a101aee5-3bb2-4c35-a963-a498c6c86d91\") " pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.701266 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.701180 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-137.ec2.internal" event={"ID":"323664165661da7466696aea846c55ee","Type":"ContainerStarted","Data":"56822e890e404637d99f48ba43f5b86333a4d938f9943da1cf2fb23750cf1723"} Sep 09 06:38:24.703072 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.703037 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-137.ec2.internal" event={"ID":"c3546eccb5a037f684d1c790f3c8d333","Type":"ContainerStarted","Data":"f7f3c0811139e6aa70ca92ee0bd768ac43ccc9a7eb81822ba0b85888362bcd48"} Sep 09 06:38:24.750619 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.750405 2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 06:38:24.760582 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.760554 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pfbsb" Sep 09 06:38:24.771850 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.771649 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5qd9z" Sep 09 06:38:24.774919 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:24.774892 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaeac7db8_f053_4218_9e15_de29565b7826.slice/crio-8295ac5db9d7fc16f7f1511bee1b9a7840d9e7d015ef9b62cfc91fb359815448 WatchSource:0}: Error finding container 8295ac5db9d7fc16f7f1511bee1b9a7840d9e7d015ef9b62cfc91fb359815448: Status 404 returned error can't find the container with id 8295ac5db9d7fc16f7f1511bee1b9a7840d9e7d015ef9b62cfc91fb359815448 Sep 09 06:38:24.779085 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.778830 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:24.783166 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.783149 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-kt6jc" Sep 09 06:38:24.788876 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.788855 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bk8ww" Sep 09 06:38:24.790151 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:24.789972 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda101aee5_3bb2_4c35_a963_a498c6c86d91.slice/crio-60d3bd3986f2677cd50f395237b152959707b324e960aa76ab06d4b204418976 WatchSource:0}: Error finding container 60d3bd3986f2677cd50f395237b152959707b324e960aa76ab06d4b204418976: Status 404 returned error can't find the container with id 60d3bd3986f2677cd50f395237b152959707b324e960aa76ab06d4b204418976 Sep 09 06:38:24.794640 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.794625 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-27vp7" Sep 09 06:38:24.801923 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:24.801896 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d5681e8_6984_40dc_b918_18483d2bed5c.slice/crio-7139aa0dbedd88f9faa718d9232175aa295e2a3dd121da24c96338e42ebd87d0 WatchSource:0}: Error finding container 7139aa0dbedd88f9faa718d9232175aa295e2a3dd121da24c96338e42ebd87d0: Status 404 returned error can't find the container with id 7139aa0dbedd88f9faa718d9232175aa295e2a3dd121da24c96338e42ebd87d0 Sep 09 06:38:24.802029 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.802011 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-w776r" Sep 09 06:38:24.805022 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.805004 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qr5vp" Sep 09 06:38:24.809031 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:24.809014 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9dd8h" Sep 09 06:38:24.810080 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:24.810040 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13c06be2_78fb_4ceb_83ee_8cc6a545b395.slice/crio-3847767b52a2411172d6bef619fd3d2334393dd41cf6540b486f9f3e6b5c1edb WatchSource:0}: Error finding container 3847767b52a2411172d6bef619fd3d2334393dd41cf6540b486f9f3e6b5c1edb: Status 404 returned error can't find the container with id 3847767b52a2411172d6bef619fd3d2334393dd41cf6540b486f9f3e6b5c1edb Sep 09 06:38:24.810995 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:24.810973 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f25c33f_2f5d_4645_990b_b9d2f201e1fc.slice/crio-d3d2455639a29c305cdf409c47cbcaac8fe467c591d3b56585d1106bd781cefc WatchSource:0}: Error finding container d3d2455639a29c305cdf409c47cbcaac8fe467c591d3b56585d1106bd781cefc: Status 404 returned error can't find the container with id d3d2455639a29c305cdf409c47cbcaac8fe467c591d3b56585d1106bd781cefc Sep 09 06:38:24.816600 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:24.816579 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode251251d_5f0e_4372_ada0_e70551d79b78.slice/crio-48e84b28dba169fbe289d3ed32973a722d85b8d07f9e464810fa48bef587dea5 WatchSource:0}: Error finding container 48e84b28dba169fbe289d3ed32973a722d85b8d07f9e464810fa48bef587dea5: Status 404 returned error can't find the container with id 48e84b28dba169fbe289d3ed32973a722d85b8d07f9e464810fa48bef587dea5 Sep 09 06:38:24.828453 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:24.828242 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1a72723_5d93_402c_8ec3_768531d2a883.slice/crio-e6db1142a455fe1563221c75ac5be2a8287c2e2879e0dcb451cc3fdc3649c474 WatchSource:0}: Error finding container e6db1142a455fe1563221c75ac5be2a8287c2e2879e0dcb451cc3fdc3649c474: Status 404 returned error can't find the container with id e6db1142a455fe1563221c75ac5be2a8287c2e2879e0dcb451cc3fdc3649c474 Sep 09 06:38:24.831122 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:24.830986 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2ab2571_e051_4e8e_baf4_ce549dbd71ba.slice/crio-bc2899787c12d84ca78b747ee299a8587a651b521318c4eed724497fb4ccd52f WatchSource:0}: Error finding container bc2899787c12d84ca78b747ee299a8587a651b521318c4eed724497fb4ccd52f: Status 404 returned error can't find the container with id bc2899787c12d84ca78b747ee299a8587a651b521318c4eed724497fb4ccd52f Sep 09 06:38:25.080067 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:25.079963 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/11156d42-6fd0-4887-9f80-6f172dcce6c1-original-pull-secret\") pod \"global-pull-secret-syncer-7rdlw\" (UID: \"11156d42-6fd0-4887-9f80-6f172dcce6c1\") " pod="kube-system/global-pull-secret-syncer-7rdlw" Sep 09 06:38:25.080067 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:25.080019 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/336552c8-ca0f-434a-8353-bc61ea118fa5-metrics-certs\") pod \"network-metrics-daemon-pkv72\" (UID: \"336552c8-ca0f-434a-8353-bc61ea118fa5\") " pod="openshift-multus/network-metrics-daemon-pkv72" Sep 09 06:38:25.080244 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:25.080174 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 06:38:25.080244 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:25.080238 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/336552c8-ca0f-434a-8353-bc61ea118fa5-metrics-certs podName:336552c8-ca0f-434a-8353-bc61ea118fa5 nodeName:}" failed. No retries permitted until 2026-09-09 06:38:26.080216055 +0000 UTC m=+3.976633475 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/336552c8-ca0f-434a-8353-bc61ea118fa5-metrics-certs") pod "network-metrics-daemon-pkv72" (UID: "336552c8-ca0f-434a-8353-bc61ea118fa5") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 06:38:25.080356 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:25.080174 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 06:38:25.080356 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:25.080337 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11156d42-6fd0-4887-9f80-6f172dcce6c1-original-pull-secret podName:11156d42-6fd0-4887-9f80-6f172dcce6c1 nodeName:}" failed. No retries permitted until 2026-09-09 06:38:26.080313967 +0000 UTC m=+3.976731383 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/11156d42-6fd0-4887-9f80-6f172dcce6c1-original-pull-secret") pod "global-pull-secret-syncer-7rdlw" (UID: "11156d42-6fd0-4887-9f80-6f172dcce6c1") : object "kube-system"/"original-pull-secret" not registered Sep 09 06:38:25.181168 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:25.181134 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhwxb\" (UniqueName: \"kubernetes.io/projected/3355475e-185e-4637-900a-5bf30d29d199-kube-api-access-fhwxb\") pod \"network-check-target-9f6j7\" (UID: \"3355475e-185e-4637-900a-5bf30d29d199\") " pod="openshift-network-diagnostics/network-check-target-9f6j7" Sep 09 06:38:25.181341 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:25.181278 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 06:38:25.181341 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:25.181295 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 06:38:25.181341 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:25.181306 2559 projected.go:196] Error preparing data for projected volume kube-api-access-fhwxb for pod openshift-network-diagnostics/network-check-target-9f6j7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 06:38:25.181503 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:25.181358 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3355475e-185e-4637-900a-5bf30d29d199-kube-api-access-fhwxb podName:3355475e-185e-4637-900a-5bf30d29d199 nodeName:}" failed. No retries permitted until 2026-09-09 06:38:26.181342685 +0000 UTC m=+4.077760108 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-fhwxb" (UniqueName: "kubernetes.io/projected/3355475e-185e-4637-900a-5bf30d29d199-kube-api-access-fhwxb") pod "network-check-target-9f6j7" (UID: "3355475e-185e-4637-900a-5bf30d29d199") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 06:38:25.503415 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:25.503310 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 06:33:24 +0000 UTC" deadline="2028-04-05 09:33:23.331080336 +0000 UTC" Sep 09 06:38:25.503415 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:25.503367 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13778h54m57.82771755s" Sep 09 06:38:25.697142 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:25.697110 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7rdlw" Sep 09 06:38:25.697293 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:25.697241 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7rdlw" podUID="11156d42-6fd0-4887-9f80-6f172dcce6c1" Sep 09 06:38:25.708578 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:25.708546 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9dd8h" event={"ID":"e2ab2571-e051-4e8e-baf4-ce549dbd71ba","Type":"ContainerStarted","Data":"bc2899787c12d84ca78b747ee299a8587a651b521318c4eed724497fb4ccd52f"} Sep 09 06:38:25.712253 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:25.712196 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-w776r" event={"ID":"e251251d-5f0e-4372-ada0-e70551d79b78","Type":"ContainerStarted","Data":"48e84b28dba169fbe289d3ed32973a722d85b8d07f9e464810fa48bef587dea5"} Sep 09 06:38:25.714582 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:25.714510 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-27vp7" event={"ID":"13c06be2-78fb-4ceb-83ee-8cc6a545b395","Type":"ContainerStarted","Data":"3847767b52a2411172d6bef619fd3d2334393dd41cf6540b486f9f3e6b5c1edb"} Sep 09 06:38:25.716959 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:25.716930 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5qd9z" event={"ID":"9c0c8eea-7803-45cf-a327-2d4c39c42360","Type":"ContainerStarted","Data":"ffc19d2eaac73906db27ba86c9a30d6af07ff9f2349f851c44dfd3821116e2e6"} Sep 09 06:38:25.719103 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:25.719079 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qr5vp" event={"ID":"f1a72723-5d93-402c-8ec3-768531d2a883","Type":"ContainerStarted","Data":"e6db1142a455fe1563221c75ac5be2a8287c2e2879e0dcb451cc3fdc3649c474"} Sep 09 06:38:25.720787 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:25.720724 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bk8ww" event={"ID":"0f25c33f-2f5d-4645-990b-b9d2f201e1fc","Type":"ContainerStarted","Data":"d3d2455639a29c305cdf409c47cbcaac8fe467c591d3b56585d1106bd781cefc"} Sep 09 06:38:25.722250 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:25.722231 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-kt6jc" event={"ID":"3d5681e8-6984-40dc-b918-18483d2bed5c","Type":"ContainerStarted","Data":"7139aa0dbedd88f9faa718d9232175aa295e2a3dd121da24c96338e42ebd87d0"} Sep 09 06:38:25.723544 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:25.723523 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" event={"ID":"a101aee5-3bb2-4c35-a963-a498c6c86d91","Type":"ContainerStarted","Data":"60d3bd3986f2677cd50f395237b152959707b324e960aa76ab06d4b204418976"} Sep 09 06:38:25.725266 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:25.725244 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pfbsb" event={"ID":"aeac7db8-f053-4218-9e15-de29565b7826","Type":"ContainerStarted","Data":"8295ac5db9d7fc16f7f1511bee1b9a7840d9e7d015ef9b62cfc91fb359815448"} Sep 09 06:38:26.088382 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:26.088348 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/11156d42-6fd0-4887-9f80-6f172dcce6c1-original-pull-secret\") pod \"global-pull-secret-syncer-7rdlw\" (UID: \"11156d42-6fd0-4887-9f80-6f172dcce6c1\") " pod="kube-system/global-pull-secret-syncer-7rdlw" Sep 09 06:38:26.088550 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:26.088403 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/336552c8-ca0f-434a-8353-bc61ea118fa5-metrics-certs\") pod \"network-metrics-daemon-pkv72\" (UID: \"336552c8-ca0f-434a-8353-bc61ea118fa5\") " pod="openshift-multus/network-metrics-daemon-pkv72" Sep 09 06:38:26.088608 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:26.088556 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 06:38:26.088676 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:26.088612 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/336552c8-ca0f-434a-8353-bc61ea118fa5-metrics-certs podName:336552c8-ca0f-434a-8353-bc61ea118fa5 nodeName:}" failed. No retries permitted until 2026-09-09 06:38:28.088594264 +0000 UTC m=+5.985011670 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/336552c8-ca0f-434a-8353-bc61ea118fa5-metrics-certs") pod "network-metrics-daemon-pkv72" (UID: "336552c8-ca0f-434a-8353-bc61ea118fa5") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 06:38:26.089014 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:26.088977 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 06:38:26.089119 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:26.089026 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11156d42-6fd0-4887-9f80-6f172dcce6c1-original-pull-secret podName:11156d42-6fd0-4887-9f80-6f172dcce6c1 nodeName:}" failed. No retries permitted until 2026-09-09 06:38:28.089011575 +0000 UTC m=+5.985428976 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/11156d42-6fd0-4887-9f80-6f172dcce6c1-original-pull-secret") pod "global-pull-secret-syncer-7rdlw" (UID: "11156d42-6fd0-4887-9f80-6f172dcce6c1") : object "kube-system"/"original-pull-secret" not registered Sep 09 06:38:26.189222 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:26.189188 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhwxb\" (UniqueName: \"kubernetes.io/projected/3355475e-185e-4637-900a-5bf30d29d199-kube-api-access-fhwxb\") pod \"network-check-target-9f6j7\" (UID: \"3355475e-185e-4637-900a-5bf30d29d199\") " pod="openshift-network-diagnostics/network-check-target-9f6j7" Sep 09 06:38:26.189389 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:26.189342 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 06:38:26.189389 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:26.189371 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 06:38:26.189389 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:26.189385 2559 projected.go:196] Error preparing data for projected volume kube-api-access-fhwxb for pod openshift-network-diagnostics/network-check-target-9f6j7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 06:38:26.189546 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:26.189457 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3355475e-185e-4637-900a-5bf30d29d199-kube-api-access-fhwxb podName:3355475e-185e-4637-900a-5bf30d29d199 nodeName:}" failed. No retries permitted until 2026-09-09 06:38:28.189429777 +0000 UTC m=+6.085847196 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-fhwxb" (UniqueName: "kubernetes.io/projected/3355475e-185e-4637-900a-5bf30d29d199-kube-api-access-fhwxb") pod "network-check-target-9f6j7" (UID: "3355475e-185e-4637-900a-5bf30d29d199") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 06:38:26.503988 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:26.503888 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 06:33:24 +0000 UTC" deadline="2028-06-07 21:17:52.865470438 +0000 UTC" Sep 09 06:38:26.503988 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:26.503922 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15302h39m26.361552308s" Sep 09 06:38:26.697932 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:26.697850 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv72" Sep 09 06:38:26.698094 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:26.697994 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkv72" podUID="336552c8-ca0f-434a-8353-bc61ea118fa5" Sep 09 06:38:26.698945 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:26.698922 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f6j7" Sep 09 06:38:26.699073 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:26.699035 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9f6j7" podUID="3355475e-185e-4637-900a-5bf30d29d199" Sep 09 06:38:26.741072 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:26.738324 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-137.ec2.internal" event={"ID":"323664165661da7466696aea846c55ee","Type":"ContainerStarted","Data":"d72f15a06d3db197a33a271da36a26aca07013df900d08410b8381899ecca437"} Sep 09 06:38:26.754960 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:26.754181 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-137.ec2.internal" podStartSLOduration=3.754161961 podStartE2EDuration="3.754161961s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 06:38:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:38:26.753926005 +0000 UTC m=+4.650343428" watchObservedRunningTime="2026-09-09 06:38:26.754161961 +0000 UTC m=+4.650579385" Sep 09 06:38:27.696917 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:27.696888 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7rdlw" Sep 09 06:38:27.697629 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:27.697015 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7rdlw" podUID="11156d42-6fd0-4887-9f80-6f172dcce6c1" Sep 09 06:38:27.764292 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:27.764211 2559 generic.go:356] "Generic (PLEG): container finished" podID="c3546eccb5a037f684d1c790f3c8d333" containerID="4dfa7bf9df6b17bd86e8be978a0c07b6f4e28e81edab3a0c2fc5f006b42a56a6" exitCode=0 Sep 09 06:38:27.765128 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:27.765101 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-137.ec2.internal" event={"ID":"c3546eccb5a037f684d1c790f3c8d333","Type":"ContainerDied","Data":"4dfa7bf9df6b17bd86e8be978a0c07b6f4e28e81edab3a0c2fc5f006b42a56a6"} Sep 09 06:38:28.107993 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:28.107915 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/11156d42-6fd0-4887-9f80-6f172dcce6c1-original-pull-secret\") pod \"global-pull-secret-syncer-7rdlw\" (UID: \"11156d42-6fd0-4887-9f80-6f172dcce6c1\") " pod="kube-system/global-pull-secret-syncer-7rdlw" Sep 09 06:38:28.107993 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:28.107970 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/336552c8-ca0f-434a-8353-bc61ea118fa5-metrics-certs\") pod \"network-metrics-daemon-pkv72\" (UID: \"336552c8-ca0f-434a-8353-bc61ea118fa5\") " pod="openshift-multus/network-metrics-daemon-pkv72" Sep 09 06:38:28.108221 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:28.108128 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 06:38:28.108221 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:28.108187 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/336552c8-ca0f-434a-8353-bc61ea118fa5-metrics-certs podName:336552c8-ca0f-434a-8353-bc61ea118fa5 nodeName:}" failed. No retries permitted until 2026-09-09 06:38:32.108169555 +0000 UTC m=+10.004586958 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/336552c8-ca0f-434a-8353-bc61ea118fa5-metrics-certs") pod "network-metrics-daemon-pkv72" (UID: "336552c8-ca0f-434a-8353-bc61ea118fa5") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 06:38:28.108567 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:28.108549 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 06:38:28.108643 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:28.108602 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11156d42-6fd0-4887-9f80-6f172dcce6c1-original-pull-secret podName:11156d42-6fd0-4887-9f80-6f172dcce6c1 nodeName:}" failed. No retries permitted until 2026-09-09 06:38:32.108586653 +0000 UTC m=+10.005004057 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/11156d42-6fd0-4887-9f80-6f172dcce6c1-original-pull-secret") pod "global-pull-secret-syncer-7rdlw" (UID: "11156d42-6fd0-4887-9f80-6f172dcce6c1") : object "kube-system"/"original-pull-secret" not registered Sep 09 06:38:28.208371 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:28.208335 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhwxb\" (UniqueName: \"kubernetes.io/projected/3355475e-185e-4637-900a-5bf30d29d199-kube-api-access-fhwxb\") pod \"network-check-target-9f6j7\" (UID: \"3355475e-185e-4637-900a-5bf30d29d199\") " pod="openshift-network-diagnostics/network-check-target-9f6j7" Sep 09 06:38:28.208546 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:28.208524 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 06:38:28.208611 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:28.208555 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 06:38:28.208611 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:28.208571 2559 projected.go:196] Error preparing data for projected volume kube-api-access-fhwxb for pod openshift-network-diagnostics/network-check-target-9f6j7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 06:38:28.208706 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:28.208630 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3355475e-185e-4637-900a-5bf30d29d199-kube-api-access-fhwxb podName:3355475e-185e-4637-900a-5bf30d29d199 nodeName:}" failed. No retries permitted until 2026-09-09 06:38:32.208610868 +0000 UTC m=+10.105028272 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-fhwxb" (UniqueName: "kubernetes.io/projected/3355475e-185e-4637-900a-5bf30d29d199-kube-api-access-fhwxb") pod "network-check-target-9f6j7" (UID: "3355475e-185e-4637-900a-5bf30d29d199") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 06:38:28.698675 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:28.697831 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv72" Sep 09 06:38:28.698675 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:28.697965 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkv72" podUID="336552c8-ca0f-434a-8353-bc61ea118fa5" Sep 09 06:38:28.698675 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:28.698425 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f6j7" Sep 09 06:38:28.698675 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:28.698520 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9f6j7" podUID="3355475e-185e-4637-900a-5bf30d29d199" Sep 09 06:38:29.697397 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:29.697151 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7rdlw" Sep 09 06:38:29.697397 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:29.697285 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7rdlw" podUID="11156d42-6fd0-4887-9f80-6f172dcce6c1" Sep 09 06:38:30.697997 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:30.697963 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv72" Sep 09 06:38:30.698381 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:30.698131 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkv72" podUID="336552c8-ca0f-434a-8353-bc61ea118fa5" Sep 09 06:38:30.698724 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:30.698500 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f6j7" Sep 09 06:38:30.698724 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:30.698585 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9f6j7" podUID="3355475e-185e-4637-900a-5bf30d29d199" Sep 09 06:38:31.696809 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:31.696772 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7rdlw" Sep 09 06:38:31.696974 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:31.696909 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7rdlw" podUID="11156d42-6fd0-4887-9f80-6f172dcce6c1" Sep 09 06:38:32.140255 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:32.139691 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/336552c8-ca0f-434a-8353-bc61ea118fa5-metrics-certs\") pod \"network-metrics-daemon-pkv72\" (UID: \"336552c8-ca0f-434a-8353-bc61ea118fa5\") " pod="openshift-multus/network-metrics-daemon-pkv72" Sep 09 06:38:32.140255 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:32.139788 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/11156d42-6fd0-4887-9f80-6f172dcce6c1-original-pull-secret\") pod \"global-pull-secret-syncer-7rdlw\" (UID: \"11156d42-6fd0-4887-9f80-6f172dcce6c1\") " pod="kube-system/global-pull-secret-syncer-7rdlw" Sep 09 06:38:32.140255 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:32.139900 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 06:38:32.140255 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:32.139963 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11156d42-6fd0-4887-9f80-6f172dcce6c1-original-pull-secret podName:11156d42-6fd0-4887-9f80-6f172dcce6c1 nodeName:}" failed. No retries permitted until 2026-09-09 06:38:40.139944944 +0000 UTC m=+18.036362358 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/11156d42-6fd0-4887-9f80-6f172dcce6c1-original-pull-secret") pod "global-pull-secret-syncer-7rdlw" (UID: "11156d42-6fd0-4887-9f80-6f172dcce6c1") : object "kube-system"/"original-pull-secret" not registered Sep 09 06:38:32.140255 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:32.139900 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 06:38:32.140255 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:32.140013 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/336552c8-ca0f-434a-8353-bc61ea118fa5-metrics-certs podName:336552c8-ca0f-434a-8353-bc61ea118fa5 nodeName:}" failed. No retries permitted until 2026-09-09 06:38:40.139999163 +0000 UTC m=+18.036416571 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/336552c8-ca0f-434a-8353-bc61ea118fa5-metrics-certs") pod "network-metrics-daemon-pkv72" (UID: "336552c8-ca0f-434a-8353-bc61ea118fa5") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 06:38:32.241130 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:32.241093 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhwxb\" (UniqueName: \"kubernetes.io/projected/3355475e-185e-4637-900a-5bf30d29d199-kube-api-access-fhwxb\") pod \"network-check-target-9f6j7\" (UID: \"3355475e-185e-4637-900a-5bf30d29d199\") " pod="openshift-network-diagnostics/network-check-target-9f6j7" Sep 09 06:38:32.241300 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:32.241258 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 06:38:32.241300 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:32.241278 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 06:38:32.241300 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:32.241290 2559 projected.go:196] Error preparing data for projected volume kube-api-access-fhwxb for pod openshift-network-diagnostics/network-check-target-9f6j7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 06:38:32.241443 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:32.241352 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3355475e-185e-4637-900a-5bf30d29d199-kube-api-access-fhwxb podName:3355475e-185e-4637-900a-5bf30d29d199 nodeName:}" failed. No retries permitted until 2026-09-09 06:38:40.241335252 +0000 UTC m=+18.137752653 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-fhwxb" (UniqueName: "kubernetes.io/projected/3355475e-185e-4637-900a-5bf30d29d199-kube-api-access-fhwxb") pod "network-check-target-9f6j7" (UID: "3355475e-185e-4637-900a-5bf30d29d199") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 06:38:32.697184 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:32.697151 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv72" Sep 09 06:38:32.697336 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:32.697280 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkv72" podUID="336552c8-ca0f-434a-8353-bc61ea118fa5" Sep 09 06:38:32.698500 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:32.698385 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f6j7" Sep 09 06:38:32.698500 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:32.698460 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9f6j7" podUID="3355475e-185e-4637-900a-5bf30d29d199" Sep 09 06:38:33.696731 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:33.696679 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7rdlw" Sep 09 06:38:33.697228 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:33.696822 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7rdlw" podUID="11156d42-6fd0-4887-9f80-6f172dcce6c1" Sep 09 06:38:34.696953 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:34.696921 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv72" Sep 09 06:38:34.697314 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:34.697032 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkv72" podUID="336552c8-ca0f-434a-8353-bc61ea118fa5" Sep 09 06:38:34.697314 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:34.697116 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f6j7" Sep 09 06:38:34.697314 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:34.697224 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9f6j7" podUID="3355475e-185e-4637-900a-5bf30d29d199" Sep 09 06:38:35.697286 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:35.697255 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7rdlw" Sep 09 06:38:35.697730 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:35.697378 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7rdlw" podUID="11156d42-6fd0-4887-9f80-6f172dcce6c1" Sep 09 06:38:36.696631 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:36.696598 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f6j7" Sep 09 06:38:36.696795 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:36.696598 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv72" Sep 09 06:38:36.697125 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:36.697085 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9f6j7" podUID="3355475e-185e-4637-900a-5bf30d29d199" Sep 09 06:38:36.697255 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:36.697230 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkv72" podUID="336552c8-ca0f-434a-8353-bc61ea118fa5" Sep 09 06:38:37.696754 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:37.696715 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7rdlw" Sep 09 06:38:37.697153 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:37.696838 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7rdlw" podUID="11156d42-6fd0-4887-9f80-6f172dcce6c1" Sep 09 06:38:38.696668 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:38.696638 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f6j7" Sep 09 06:38:38.696845 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:38.696638 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv72" Sep 09 06:38:38.696845 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:38.696765 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9f6j7" podUID="3355475e-185e-4637-900a-5bf30d29d199" Sep 09 06:38:38.697266 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:38.696881 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkv72" podUID="336552c8-ca0f-434a-8353-bc61ea118fa5" Sep 09 06:38:39.696364 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:39.696332 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7rdlw" Sep 09 06:38:39.696577 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:39.696460 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7rdlw" podUID="11156d42-6fd0-4887-9f80-6f172dcce6c1" Sep 09 06:38:40.199741 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:40.199706 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/11156d42-6fd0-4887-9f80-6f172dcce6c1-original-pull-secret\") pod \"global-pull-secret-syncer-7rdlw\" (UID: \"11156d42-6fd0-4887-9f80-6f172dcce6c1\") " pod="kube-system/global-pull-secret-syncer-7rdlw" Sep 09 06:38:40.200269 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:40.199759 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/336552c8-ca0f-434a-8353-bc61ea118fa5-metrics-certs\") pod \"network-metrics-daemon-pkv72\" (UID: \"336552c8-ca0f-434a-8353-bc61ea118fa5\") " pod="openshift-multus/network-metrics-daemon-pkv72" Sep 09 06:38:40.200269 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:40.199861 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 06:38:40.200269 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:40.199876 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 06:38:40.200269 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:40.199939 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11156d42-6fd0-4887-9f80-6f172dcce6c1-original-pull-secret podName:11156d42-6fd0-4887-9f80-6f172dcce6c1 nodeName:}" failed. No retries permitted until 2026-09-09 06:38:56.199922073 +0000 UTC m=+34.096339473 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/11156d42-6fd0-4887-9f80-6f172dcce6c1-original-pull-secret") pod "global-pull-secret-syncer-7rdlw" (UID: "11156d42-6fd0-4887-9f80-6f172dcce6c1") : object "kube-system"/"original-pull-secret" not registered Sep 09 06:38:40.200269 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:40.199962 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/336552c8-ca0f-434a-8353-bc61ea118fa5-metrics-certs podName:336552c8-ca0f-434a-8353-bc61ea118fa5 nodeName:}" failed. No retries permitted until 2026-09-09 06:38:56.199948766 +0000 UTC m=+34.096366166 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/336552c8-ca0f-434a-8353-bc61ea118fa5-metrics-certs") pod "network-metrics-daemon-pkv72" (UID: "336552c8-ca0f-434a-8353-bc61ea118fa5") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 06:38:40.300166 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:40.300136 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhwxb\" (UniqueName: \"kubernetes.io/projected/3355475e-185e-4637-900a-5bf30d29d199-kube-api-access-fhwxb\") pod \"network-check-target-9f6j7\" (UID: \"3355475e-185e-4637-900a-5bf30d29d199\") " pod="openshift-network-diagnostics/network-check-target-9f6j7" Sep 09 06:38:40.300347 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:40.300263 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 06:38:40.300347 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:40.300279 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 06:38:40.300347 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:40.300288 2559 projected.go:196] Error preparing data for projected volume kube-api-access-fhwxb for pod openshift-network-diagnostics/network-check-target-9f6j7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 06:38:40.300347 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:40.300341 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3355475e-185e-4637-900a-5bf30d29d199-kube-api-access-fhwxb podName:3355475e-185e-4637-900a-5bf30d29d199 nodeName:}" failed. No retries permitted until 2026-09-09 06:38:56.300322742 +0000 UTC m=+34.196740162 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-fhwxb" (UniqueName: "kubernetes.io/projected/3355475e-185e-4637-900a-5bf30d29d199-kube-api-access-fhwxb") pod "network-check-target-9f6j7" (UID: "3355475e-185e-4637-900a-5bf30d29d199") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 06:38:40.697068 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:40.697018 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv72" Sep 09 06:38:40.697252 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:40.697156 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkv72" podUID="336552c8-ca0f-434a-8353-bc61ea118fa5" Sep 09 06:38:40.697252 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:40.697208 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f6j7" Sep 09 06:38:40.697372 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:40.697300 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9f6j7" podUID="3355475e-185e-4637-900a-5bf30d29d199" Sep 09 06:38:41.697377 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:41.697341 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7rdlw" Sep 09 06:38:41.697801 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:41.697470 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7rdlw" podUID="11156d42-6fd0-4887-9f80-6f172dcce6c1" Sep 09 06:38:42.697584 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:42.697554 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv72" Sep 09 06:38:42.697945 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:42.697639 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkv72" podUID="336552c8-ca0f-434a-8353-bc61ea118fa5" Sep 09 06:38:42.697945 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:42.697666 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f6j7" Sep 09 06:38:42.697945 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:42.697707 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9f6j7" podUID="3355475e-185e-4637-900a-5bf30d29d199" Sep 09 06:38:43.697534 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:43.697361 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7rdlw" Sep 09 06:38:43.697645 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:43.697612 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7rdlw" podUID="11156d42-6fd0-4887-9f80-6f172dcce6c1" Sep 09 06:38:43.798276 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:43.798109 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-137.ec2.internal" event={"ID":"c3546eccb5a037f684d1c790f3c8d333","Type":"ContainerStarted","Data":"66a5e2358b58922f97cd6745a4dc2d0041c75ef7e258baaa4f85f586e28ff9ba"} Sep 09 06:38:43.799435 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:43.799414 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qr5vp" event={"ID":"f1a72723-5d93-402c-8ec3-768531d2a883","Type":"ContainerStarted","Data":"42ca1de9e7e156e448b23f7b90c2a85fae4652bc97a30dcecff927bddf5bba41"} Sep 09 06:38:43.800666 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:43.800645 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bk8ww" event={"ID":"0f25c33f-2f5d-4645-990b-b9d2f201e1fc","Type":"ContainerStarted","Data":"43e7e2f47e8cbcdc01a96b0d5ebc1ed686e2c305ab49c80762d152813ed65bfc"} Sep 09 06:38:43.802890 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:43.802759 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-kt6jc" event={"ID":"3d5681e8-6984-40dc-b918-18483d2bed5c","Type":"ContainerStarted","Data":"c4e3da65f3bf7d97a9fcc46ff64555a6e92371eb5ea6caa79273d720646d165b"} Sep 09 06:38:43.806108 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:43.806084 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" event={"ID":"a101aee5-3bb2-4c35-a963-a498c6c86d91","Type":"ContainerStarted","Data":"8cff911ad98e3ae961b63e3ec61f8be4c5f33d3725bdf3601b7f3db7fdac5f47"} Sep 09 06:38:43.806199 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:43.806147 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" event={"ID":"a101aee5-3bb2-4c35-a963-a498c6c86d91","Type":"ContainerStarted","Data":"d4f64cbf9f33ee7d3f5168e1185eddf962d0e2d6ed12ab7f2b377d5b17e0ce23"} Sep 09 06:38:43.807758 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:43.807735 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pfbsb" event={"ID":"aeac7db8-f053-4218-9e15-de29565b7826","Type":"ContainerStarted","Data":"f926aa3ef0af26831b9ee4ee232c03899509cc9cc63331fee3a711f9542db794"} Sep 09 06:38:43.809372 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:43.809344 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9dd8h" event={"ID":"e2ab2571-e051-4e8e-baf4-ce549dbd71ba","Type":"ContainerStarted","Data":"8d9075e4318cfed770a0006a003adb6bcdf2d1a7796b62508f0abcb33e560592"} Sep 09 06:38:43.811416 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:43.811392 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-w776r" event={"ID":"e251251d-5f0e-4372-ada0-e70551d79b78","Type":"ContainerStarted","Data":"2065f42d4ac395a7dab8fa1fba46a369f6eb78e27edaf171d606105b0b7eca80"} Sep 09 06:38:43.813728 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:43.813679 2559 generic.go:356] "Generic (PLEG): container finished" podID="13c06be2-78fb-4ceb-83ee-8cc6a545b395" containerID="4bb89eb6c58c2a478834f34a5956336af164dafbe6add557939fd47d24726e97" exitCode=0 Sep 09 06:38:43.813807 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:43.813733 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-27vp7" event={"ID":"13c06be2-78fb-4ceb-83ee-8cc6a545b395","Type":"ContainerDied","Data":"4bb89eb6c58c2a478834f34a5956336af164dafbe6add557939fd47d24726e97"} Sep 09 06:38:43.823722 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:43.823688 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-bk8ww" podStartSLOduration=3.8143127310000002 podStartE2EDuration="21.823669623s" totalImagesPullingTime="18.009356892s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 06:38:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:38:43.823531541 +0000 UTC m=+21.719948964" watchObservedRunningTime="2026-09-09 06:38:43.823669623 +0000 UTC m=+21.720087039" Sep 09 06:38:43.823824 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:43.823781 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-137.ec2.internal" podStartSLOduration=20.82377466 podStartE2EDuration="20.82377466s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 06:38:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:38:43.811472332 +0000 UTC m=+21.707889754" watchObservedRunningTime="2026-09-09 06:38:43.82377466 +0000 UTC m=+21.720192079" Sep 09 06:38:43.855654 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:43.855605 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-w776r" podStartSLOduration=3.854416773 podStartE2EDuration="21.855592492s" totalImagesPullingTime="18.001175719s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 06:38:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:38:43.84015204 +0000 UTC m=+21.736569463" watchObservedRunningTime="2026-09-09 06:38:43.855592492 +0000 UTC m=+21.752009944" Sep 09 06:38:43.855784 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:43.855771 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-pfbsb" podStartSLOduration=3.758142749 podStartE2EDuration="21.855767512s" totalImagesPullingTime="18.097624763s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 06:38:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:38:43.855588167 +0000 UTC m=+21.752005589" watchObservedRunningTime="2026-09-09 06:38:43.855767512 +0000 UTC m=+21.752184933" Sep 09 06:38:43.859659 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:43.859633 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-kt6jc" Sep 09 06:38:43.860198 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:43.860178 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-kt6jc" Sep 09 06:38:43.898005 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:43.897963 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-kt6jc" podStartSLOduration=3.876789516 podStartE2EDuration="21.897949148s" totalImagesPullingTime="18.021159632s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:38:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:38:43.877233604 +0000 UTC m=+21.773651027" watchObservedRunningTime="2026-09-09 06:38:43.897949148 +0000 UTC m=+21.794366570" Sep 09 06:38:43.898226 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:43.898196 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qr5vp" podStartSLOduration=3.904175104 podStartE2EDuration="21.898188707s" totalImagesPullingTime="17.994013603s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 06:38:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:38:43.89772322 +0000 UTC m=+21.794140643" watchObservedRunningTime="2026-09-09 06:38:43.898188707 +0000 UTC m=+21.794606129" Sep 09 06:38:44.133254 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:44.133228 2559 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 09 06:38:44.517621 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:44.517483 2559 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-09T06:38:44.133249701Z","UUID":"87fa05fc-4bea-4ea2-b399-e5a03a3ce056","Handler":null,"Name":"","Endpoint":""} Sep 09 06:38:44.519069 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:44.519038 2559 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 09 06:38:44.519187 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:44.519085 2559 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 09 06:38:44.696527 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:44.696491 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv72" Sep 09 06:38:44.696527 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:44.696528 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f6j7" Sep 09 06:38:44.696763 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:44.696620 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkv72" podUID="336552c8-ca0f-434a-8353-bc61ea118fa5" Sep 09 06:38:44.696825 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:44.696744 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9f6j7" podUID="3355475e-185e-4637-900a-5bf30d29d199" Sep 09 06:38:44.820756 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:44.820652 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" event={"ID":"a101aee5-3bb2-4c35-a963-a498c6c86d91","Type":"ContainerStarted","Data":"805804a0791341bba2e69d167b1b4973502b0d4295ea9acf49fa7e065e76d5d4"} Sep 09 06:38:44.821654 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:44.820753 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" event={"ID":"a101aee5-3bb2-4c35-a963-a498c6c86d91","Type":"ContainerStarted","Data":"b659687b5dcadea78b4dc7ccb16cbe6613759d2bf018a1df66e29a45e8f1204c"} Sep 09 06:38:44.821654 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:44.820801 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" event={"ID":"a101aee5-3bb2-4c35-a963-a498c6c86d91","Type":"ContainerStarted","Data":"93d5e83da23c2fe7cc073ee00bb3a16fb2bb23207a59a118dfcf7a4c6685db11"} Sep 09 06:38:44.821654 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:44.820850 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" event={"ID":"a101aee5-3bb2-4c35-a963-a498c6c86d91","Type":"ContainerStarted","Data":"f1cde7fda953d82a54c38bb31de35863c4e1bc2863d26fd1845c552b24c82157"} Sep 09 06:38:44.823165 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:44.823137 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9dd8h" event={"ID":"e2ab2571-e051-4e8e-baf4-ce549dbd71ba","Type":"ContainerStarted","Data":"ef294279de890d85db464973b5009146fe2e795ba19e17cc45c88124537cad50"} Sep 09 06:38:44.825224 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:44.825140 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5qd9z" event={"ID":"9c0c8eea-7803-45cf-a327-2d4c39c42360","Type":"ContainerStarted","Data":"7790ebe87dcdbfeab2b4f65449e18cbda2d9351ec34aacd5163c8b4336e6cc56"} Sep 09 06:38:44.825629 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:44.825608 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-kt6jc" Sep 09 06:38:44.826422 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:44.826403 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-kt6jc" Sep 09 06:38:44.842242 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:44.842195 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-5qd9z" podStartSLOduration=4.803924858 podStartE2EDuration="22.842179038s" totalImagesPullingTime="18.03825418s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 06:38:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:38:44.84215049 +0000 UTC m=+22.738567914" watchObservedRunningTime="2026-09-09 06:38:44.842179038 +0000 UTC m=+22.738596461" Sep 09 06:38:45.696811 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:45.696600 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7rdlw" Sep 09 06:38:45.696992 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:45.696891 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7rdlw" podUID="11156d42-6fd0-4887-9f80-6f172dcce6c1" Sep 09 06:38:45.829978 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:45.829941 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9dd8h" event={"ID":"e2ab2571-e051-4e8e-baf4-ce549dbd71ba","Type":"ContainerStarted","Data":"c1aa2b11ba21333c9d5abc6a1058fc8ac07f4452ff99df3f772de12e8fe13a39"} Sep 09 06:38:45.848556 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:45.848501 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9dd8h" podStartSLOduration=2.850433151 podStartE2EDuration="22.848484476s" totalImagesPullingTime="19.998051325s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 06:38:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:38:45.847933857 +0000 UTC m=+23.744351263" watchObservedRunningTime="2026-09-09 06:38:45.848484476 +0000 UTC m=+23.744901897" Sep 09 06:38:46.696970 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:46.696939 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv72" Sep 09 06:38:46.697194 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:46.697095 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkv72" podUID="336552c8-ca0f-434a-8353-bc61ea118fa5" Sep 09 06:38:46.697194 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:46.697145 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f6j7" Sep 09 06:38:46.697309 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:46.697259 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9f6j7" podUID="3355475e-185e-4637-900a-5bf30d29d199" Sep 09 06:38:46.836821 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:46.836782 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" event={"ID":"a101aee5-3bb2-4c35-a963-a498c6c86d91","Type":"ContainerStarted","Data":"cfdde28488be594f7cee186d745384805f47033d3b910bd63f81e219e6603929"} Sep 09 06:38:47.697153 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:47.697106 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7rdlw" Sep 09 06:38:47.697317 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:47.697228 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7rdlw" podUID="11156d42-6fd0-4887-9f80-6f172dcce6c1" Sep 09 06:38:48.697211 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:48.697015 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv72" Sep 09 06:38:48.697547 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:48.697096 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f6j7" Sep 09 06:38:48.697547 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:48.697312 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkv72" podUID="336552c8-ca0f-434a-8353-bc61ea118fa5" Sep 09 06:38:48.697547 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:48.697354 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9f6j7" podUID="3355475e-185e-4637-900a-5bf30d29d199" Sep 09 06:38:48.842576 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:48.842541 2559 generic.go:356] "Generic (PLEG): container finished" podID="13c06be2-78fb-4ceb-83ee-8cc6a545b395" containerID="23a4f6984f13ba023334cc68fa052fc5f366e4f1bb6987988330874eeab80893" exitCode=0 Sep 09 06:38:48.842718 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:48.842627 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-27vp7" event={"ID":"13c06be2-78fb-4ceb-83ee-8cc6a545b395","Type":"ContainerDied","Data":"23a4f6984f13ba023334cc68fa052fc5f366e4f1bb6987988330874eeab80893"} Sep 09 06:38:48.847031 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:48.847003 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" event={"ID":"a101aee5-3bb2-4c35-a963-a498c6c86d91","Type":"ContainerStarted","Data":"0001411e52a63918e42f0a04c28e3d54f2766337a0f841c18c53dd5e84845b83"} Sep 09 06:38:48.847340 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:48.847320 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:48.847417 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:48.847380 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:48.861728 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:48.861709 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:48.888417 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:48.886899 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" podStartSLOduration=8.014199817 podStartE2EDuration="26.886883205s" totalImagesPullingTime="18.872683388s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 06:38:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:38:48.886754466 +0000 UTC m=+26.783171889" watchObservedRunningTime="2026-09-09 06:38:48.886883205 +0000 UTC m=+26.783300619" Sep 09 06:38:49.696354 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:49.696325 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7rdlw" Sep 09 06:38:49.696539 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:49.696447 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7rdlw" podUID="11156d42-6fd0-4887-9f80-6f172dcce6c1" Sep 09 06:38:49.852135 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:49.852102 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-27vp7" event={"ID":"13c06be2-78fb-4ceb-83ee-8cc6a545b395","Type":"ContainerStarted","Data":"aa705d48a103e1ca1bafeab928f33aaa62f25ea71b98b7296ae28bcb014a5778"} Sep 09 06:38:49.852715 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:49.852695 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:49.874554 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:49.874533 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:38:50.188284 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:50.188253 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9f6j7"] Sep 09 06:38:50.188455 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:50.188376 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f6j7" Sep 09 06:38:50.188518 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:50.188479 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9f6j7" podUID="3355475e-185e-4637-900a-5bf30d29d199" Sep 09 06:38:50.191739 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:50.191712 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pkv72"] Sep 09 06:38:50.191849 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:50.191806 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv72" Sep 09 06:38:50.191916 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:50.191895 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkv72" podUID="336552c8-ca0f-434a-8353-bc61ea118fa5" Sep 09 06:38:50.192571 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:50.192545 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7rdlw"] Sep 09 06:38:50.192660 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:50.192616 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7rdlw" Sep 09 06:38:50.192737 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:50.192697 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7rdlw" podUID="11156d42-6fd0-4887-9f80-6f172dcce6c1" Sep 09 06:38:50.856667 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:50.856584 2559 generic.go:356] "Generic (PLEG): container finished" podID="13c06be2-78fb-4ceb-83ee-8cc6a545b395" containerID="aa705d48a103e1ca1bafeab928f33aaa62f25ea71b98b7296ae28bcb014a5778" exitCode=0 Sep 09 06:38:50.857172 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:50.856666 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-27vp7" event={"ID":"13c06be2-78fb-4ceb-83ee-8cc6a545b395","Type":"ContainerDied","Data":"aa705d48a103e1ca1bafeab928f33aaa62f25ea71b98b7296ae28bcb014a5778"} Sep 09 06:38:51.697279 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:51.697214 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f6j7" Sep 09 06:38:51.697454 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:51.697340 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7rdlw" Sep 09 06:38:51.697454 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:51.697344 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9f6j7" podUID="3355475e-185e-4637-900a-5bf30d29d199" Sep 09 06:38:51.697454 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:51.697385 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv72" Sep 09 06:38:51.697615 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:51.697481 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7rdlw" podUID="11156d42-6fd0-4887-9f80-6f172dcce6c1" Sep 09 06:38:51.697615 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:51.697553 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkv72" podUID="336552c8-ca0f-434a-8353-bc61ea118fa5" Sep 09 06:38:51.861687 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:51.861446 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-27vp7" event={"ID":"13c06be2-78fb-4ceb-83ee-8cc6a545b395","Type":"ContainerStarted","Data":"5970c25192d400b3f5c637e79e87783b65f328040167473baeb9a118295105d3"} Sep 09 06:38:52.868782 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:52.868744 2559 generic.go:356] "Generic (PLEG): container finished" podID="13c06be2-78fb-4ceb-83ee-8cc6a545b395" containerID="5970c25192d400b3f5c637e79e87783b65f328040167473baeb9a118295105d3" exitCode=0 Sep 09 06:38:52.869254 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:52.868814 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-27vp7" event={"ID":"13c06be2-78fb-4ceb-83ee-8cc6a545b395","Type":"ContainerDied","Data":"5970c25192d400b3f5c637e79e87783b65f328040167473baeb9a118295105d3"} Sep 09 06:38:53.696499 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:53.696460 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f6j7" Sep 09 06:38:53.696499 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:53.696492 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv72" Sep 09 06:38:53.696499 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:53.696498 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7rdlw" Sep 09 06:38:53.696749 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:53.696584 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9f6j7" podUID="3355475e-185e-4637-900a-5bf30d29d199" Sep 09 06:38:53.696749 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:53.696716 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkv72" podUID="336552c8-ca0f-434a-8353-bc61ea118fa5" Sep 09 06:38:53.696826 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:53.696802 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7rdlw" podUID="11156d42-6fd0-4887-9f80-6f172dcce6c1" Sep 09 06:38:55.697347 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:55.697313 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f6j7" Sep 09 06:38:55.697785 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:55.697450 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7rdlw" Sep 09 06:38:55.697785 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:55.697456 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9f6j7" podUID="3355475e-185e-4637-900a-5bf30d29d199" Sep 09 06:38:55.697785 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:55.697561 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7rdlw" podUID="11156d42-6fd0-4887-9f80-6f172dcce6c1" Sep 09 06:38:55.697785 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:55.697458 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv72" Sep 09 06:38:55.697785 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:55.697659 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkv72" podUID="336552c8-ca0f-434a-8353-bc61ea118fa5" Sep 09 06:38:55.846697 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:55.846615 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-137.ec2.internal" event="NodeReady" Sep 09 06:38:55.846853 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:55.846760 2559 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 09 06:38:55.882074 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:55.882029 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-798d7b96c-6srkk"] Sep 09 06:38:55.896492 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:55.896455 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-5x9qz"] Sep 09 06:38:55.896623 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:55.896577 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:38:55.900175 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:55.899875 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9h4tb\"" Sep 09 06:38:55.900175 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:55.900134 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 09 06:38:55.900902 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:55.900267 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 09 06:38:55.901531 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:55.900988 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 09 06:38:55.905042 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:55.904862 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 09 06:38:55.914411 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:55.914389 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-5x9qz"] Sep 09 06:38:55.914522 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:55.914427 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-798d7b96c-6srkk"] Sep 09 06:38:55.914522 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:55.914452 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4lksd"] Sep 09 06:38:55.914679 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:55.914659 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-5x9qz" Sep 09 06:38:55.916965 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:55.916943 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Sep 09 06:38:55.917096 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:55.916988 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Sep 09 06:38:55.917166 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:55.916943 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-pb4cn\"" Sep 09 06:38:55.936727 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:55.936687 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-g54h7"] Sep 09 06:38:55.938279 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:55.938254 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4lksd" Sep 09 06:38:55.941177 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:55.941139 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 09 06:38:55.941286 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:55.941252 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 09 06:38:55.941505 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:55.941488 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 09 06:38:55.941807 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:55.941788 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jc29b\"" Sep 09 06:38:55.958441 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:55.958421 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4lksd"] Sep 09 06:38:55.958527 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:55.958457 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-g54h7"] Sep 09 06:38:55.958580 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:55.958548 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-g54h7" Sep 09 06:38:55.960716 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:55.960698 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 09 06:38:55.960897 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:55.960865 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4pf4g\"" Sep 09 06:38:55.960992 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:55.960898 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 09 06:38:56.020987 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.020961 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fe89f009-e9d3-404a-b39a-895d216f3c22-ca-trust-extracted\") pod \"image-registry-798d7b96c-6srkk\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:38:56.021140 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.020997 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jswlk\" (UniqueName: \"kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-kube-api-access-jswlk\") pod \"image-registry-798d7b96c-6srkk\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:38:56.021140 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.021025 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/635a2ebb-6998-49e2-8e4d-088725c2c079-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-5x9qz\" (UID: \"635a2ebb-6998-49e2-8e4d-088725c2c079\") " pod="openshift-network-console/networking-console-plugin-79977f9844-5x9qz" Sep 09 06:38:56.021140 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.021078 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fe89f009-e9d3-404a-b39a-895d216f3c22-image-registry-private-configuration\") pod \"image-registry-798d7b96c-6srkk\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:38:56.021140 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.021112 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-certificates\") pod \"image-registry-798d7b96c-6srkk\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:38:56.021140 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.021134 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fe89f009-e9d3-404a-b39a-895d216f3c22-installation-pull-secrets\") pod \"image-registry-798d7b96c-6srkk\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:38:56.021389 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.021196 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/635a2ebb-6998-49e2-8e4d-088725c2c079-nginx-conf\") pod \"networking-console-plugin-79977f9844-5x9qz\" (UID: \"635a2ebb-6998-49e2-8e4d-088725c2c079\") " pod="openshift-network-console/networking-console-plugin-79977f9844-5x9qz" Sep 09 06:38:56.021389 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.021238 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fe89f009-e9d3-404a-b39a-895d216f3c22-trusted-ca\") pod \"image-registry-798d7b96c-6srkk\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:38:56.021389 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.021261 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-bound-sa-token\") pod \"image-registry-798d7b96c-6srkk\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:38:56.021389 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.021293 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-tls\") pod \"image-registry-798d7b96c-6srkk\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:38:56.122390 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.122296 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-tls\") pod \"image-registry-798d7b96c-6srkk\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:38:56.122390 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.122345 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcr5s\" (UniqueName: \"kubernetes.io/projected/5e0761f7-70d1-4952-b645-1b99f46e3fc7-kube-api-access-tcr5s\") pod \"dns-default-g54h7\" (UID: \"5e0761f7-70d1-4952-b645-1b99f46e3fc7\") " pod="openshift-dns/dns-default-g54h7" Sep 09 06:38:56.122390 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.122380 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fe89f009-e9d3-404a-b39a-895d216f3c22-ca-trust-extracted\") pod \"image-registry-798d7b96c-6srkk\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:38:56.122649 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.122450 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jswlk\" (UniqueName: \"kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-kube-api-access-jswlk\") pod \"image-registry-798d7b96c-6srkk\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:38:56.122649 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:56.122491 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 06:38:56.122649 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:56.122513 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-798d7b96c-6srkk: secret "image-registry-tls" not found Sep 09 06:38:56.122649 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:56.122556 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 09 06:38:56.122649 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:56.122584 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-tls podName:fe89f009-e9d3-404a-b39a-895d216f3c22 nodeName:}" failed. No retries permitted until 2026-09-09 06:38:56.622563049 +0000 UTC m=+34.518980467 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-tls") pod "image-registry-798d7b96c-6srkk" (UID: "fe89f009-e9d3-404a-b39a-895d216f3c22") : secret "image-registry-tls" not found Sep 09 06:38:56.122649 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:56.122618 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/635a2ebb-6998-49e2-8e4d-088725c2c079-networking-console-plugin-cert podName:635a2ebb-6998-49e2-8e4d-088725c2c079 nodeName:}" failed. No retries permitted until 2026-09-09 06:38:56.622600543 +0000 UTC m=+34.519017943 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/635a2ebb-6998-49e2-8e4d-088725c2c079-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-5x9qz" (UID: "635a2ebb-6998-49e2-8e4d-088725c2c079") : secret "networking-console-plugin-cert" not found Sep 09 06:38:56.122649 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.122489 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/635a2ebb-6998-49e2-8e4d-088725c2c079-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-5x9qz\" (UID: \"635a2ebb-6998-49e2-8e4d-088725c2c079\") " pod="openshift-network-console/networking-console-plugin-79977f9844-5x9qz" Sep 09 06:38:56.122987 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.122657 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwgdn\" (UniqueName: \"kubernetes.io/projected/7e3c731f-3b9b-445d-a6d9-38c20b30568e-kube-api-access-mwgdn\") pod \"ingress-canary-4lksd\" (UID: \"7e3c731f-3b9b-445d-a6d9-38c20b30568e\") " pod="openshift-ingress-canary/ingress-canary-4lksd" Sep 09 06:38:56.122987 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.122686 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e0761f7-70d1-4952-b645-1b99f46e3fc7-metrics-tls\") pod \"dns-default-g54h7\" (UID: \"5e0761f7-70d1-4952-b645-1b99f46e3fc7\") " pod="openshift-dns/dns-default-g54h7" Sep 09 06:38:56.122987 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.122718 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fe89f009-e9d3-404a-b39a-895d216f3c22-image-registry-private-configuration\") pod \"image-registry-798d7b96c-6srkk\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:38:56.122987 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.122736 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5e0761f7-70d1-4952-b645-1b99f46e3fc7-config-volume\") pod \"dns-default-g54h7\" (UID: \"5e0761f7-70d1-4952-b645-1b99f46e3fc7\") " pod="openshift-dns/dns-default-g54h7" Sep 09 06:38:56.122987 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.122763 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-certificates\") pod \"image-registry-798d7b96c-6srkk\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:38:56.122987 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.122764 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fe89f009-e9d3-404a-b39a-895d216f3c22-ca-trust-extracted\") pod \"image-registry-798d7b96c-6srkk\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:38:56.122987 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.122785 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fe89f009-e9d3-404a-b39a-895d216f3c22-installation-pull-secrets\") pod \"image-registry-798d7b96c-6srkk\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:38:56.122987 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.122916 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/635a2ebb-6998-49e2-8e4d-088725c2c079-nginx-conf\") pod \"networking-console-plugin-79977f9844-5x9qz\" (UID: \"635a2ebb-6998-49e2-8e4d-088725c2c079\") " pod="openshift-network-console/networking-console-plugin-79977f9844-5x9qz" Sep 09 06:38:56.122987 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.122957 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e0761f7-70d1-4952-b645-1b99f46e3fc7-tmp-dir\") pod \"dns-default-g54h7\" (UID: \"5e0761f7-70d1-4952-b645-1b99f46e3fc7\") " pod="openshift-dns/dns-default-g54h7" Sep 09 06:38:56.122987 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.122984 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7e3c731f-3b9b-445d-a6d9-38c20b30568e-cert\") pod \"ingress-canary-4lksd\" (UID: \"7e3c731f-3b9b-445d-a6d9-38c20b30568e\") " pod="openshift-ingress-canary/ingress-canary-4lksd" Sep 09 06:38:56.123475 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.123024 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fe89f009-e9d3-404a-b39a-895d216f3c22-trusted-ca\") pod \"image-registry-798d7b96c-6srkk\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:38:56.123475 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.123071 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-bound-sa-token\") pod \"image-registry-798d7b96c-6srkk\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:38:56.123475 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.123303 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-certificates\") pod \"image-registry-798d7b96c-6srkk\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:38:56.123966 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.123942 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/635a2ebb-6998-49e2-8e4d-088725c2c079-nginx-conf\") pod \"networking-console-plugin-79977f9844-5x9qz\" (UID: \"635a2ebb-6998-49e2-8e4d-088725c2c079\") " pod="openshift-network-console/networking-console-plugin-79977f9844-5x9qz" Sep 09 06:38:56.124258 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.124222 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fe89f009-e9d3-404a-b39a-895d216f3c22-trusted-ca\") pod \"image-registry-798d7b96c-6srkk\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:38:56.127173 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.127125 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fe89f009-e9d3-404a-b39a-895d216f3c22-installation-pull-secrets\") pod \"image-registry-798d7b96c-6srkk\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:38:56.127173 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.127128 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fe89f009-e9d3-404a-b39a-895d216f3c22-image-registry-private-configuration\") pod \"image-registry-798d7b96c-6srkk\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:38:56.131080 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.131029 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jswlk\" (UniqueName: \"kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-kube-api-access-jswlk\") pod \"image-registry-798d7b96c-6srkk\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:38:56.131235 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.131216 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-bound-sa-token\") pod \"image-registry-798d7b96c-6srkk\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:38:56.223439 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.223396 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcr5s\" (UniqueName: \"kubernetes.io/projected/5e0761f7-70d1-4952-b645-1b99f46e3fc7-kube-api-access-tcr5s\") pod \"dns-default-g54h7\" (UID: \"5e0761f7-70d1-4952-b645-1b99f46e3fc7\") " pod="openshift-dns/dns-default-g54h7" Sep 09 06:38:56.223640 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.223471 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwgdn\" (UniqueName: \"kubernetes.io/projected/7e3c731f-3b9b-445d-a6d9-38c20b30568e-kube-api-access-mwgdn\") pod \"ingress-canary-4lksd\" (UID: \"7e3c731f-3b9b-445d-a6d9-38c20b30568e\") " pod="openshift-ingress-canary/ingress-canary-4lksd" Sep 09 06:38:56.223640 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.223499 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e0761f7-70d1-4952-b645-1b99f46e3fc7-metrics-tls\") pod \"dns-default-g54h7\" (UID: \"5e0761f7-70d1-4952-b645-1b99f46e3fc7\") " pod="openshift-dns/dns-default-g54h7" Sep 09 06:38:56.223640 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.223533 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5e0761f7-70d1-4952-b645-1b99f46e3fc7-config-volume\") pod \"dns-default-g54h7\" (UID: \"5e0761f7-70d1-4952-b645-1b99f46e3fc7\") " pod="openshift-dns/dns-default-g54h7" Sep 09 06:38:56.223640 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.223567 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/11156d42-6fd0-4887-9f80-6f172dcce6c1-original-pull-secret\") pod \"global-pull-secret-syncer-7rdlw\" (UID: \"11156d42-6fd0-4887-9f80-6f172dcce6c1\") " pod="kube-system/global-pull-secret-syncer-7rdlw" Sep 09 06:38:56.223640 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.223601 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/336552c8-ca0f-434a-8353-bc61ea118fa5-metrics-certs\") pod \"network-metrics-daemon-pkv72\" (UID: \"336552c8-ca0f-434a-8353-bc61ea118fa5\") " pod="openshift-multus/network-metrics-daemon-pkv72" Sep 09 06:38:56.223640 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.223628 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e0761f7-70d1-4952-b645-1b99f46e3fc7-tmp-dir\") pod \"dns-default-g54h7\" (UID: \"5e0761f7-70d1-4952-b645-1b99f46e3fc7\") " pod="openshift-dns/dns-default-g54h7" Sep 09 06:38:56.223887 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.223647 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7e3c731f-3b9b-445d-a6d9-38c20b30568e-cert\") pod \"ingress-canary-4lksd\" (UID: \"7e3c731f-3b9b-445d-a6d9-38c20b30568e\") " pod="openshift-ingress-canary/ingress-canary-4lksd" Sep 09 06:38:56.223887 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:56.223702 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 06:38:56.223887 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:56.223759 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 06:38:56.223887 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:56.223781 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11156d42-6fd0-4887-9f80-6f172dcce6c1-original-pull-secret podName:11156d42-6fd0-4887-9f80-6f172dcce6c1 nodeName:}" failed. No retries permitted until 2026-09-09 06:39:28.223762419 +0000 UTC m=+66.120179825 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/11156d42-6fd0-4887-9f80-6f172dcce6c1-original-pull-secret") pod "global-pull-secret-syncer-7rdlw" (UID: "11156d42-6fd0-4887-9f80-6f172dcce6c1") : object "kube-system"/"original-pull-secret" not registered Sep 09 06:38:56.223887 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:56.223804 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/336552c8-ca0f-434a-8353-bc61ea118fa5-metrics-certs podName:336552c8-ca0f-434a-8353-bc61ea118fa5 nodeName:}" failed. No retries permitted until 2026-09-09 06:39:28.223793179 +0000 UTC m=+66.120210597 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/336552c8-ca0f-434a-8353-bc61ea118fa5-metrics-certs") pod "network-metrics-daemon-pkv72" (UID: "336552c8-ca0f-434a-8353-bc61ea118fa5") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 06:38:56.224046 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:56.223924 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 06:38:56.224046 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:56.223978 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e0761f7-70d1-4952-b645-1b99f46e3fc7-metrics-tls podName:5e0761f7-70d1-4952-b645-1b99f46e3fc7 nodeName:}" failed. No retries permitted until 2026-09-09 06:38:56.723962585 +0000 UTC m=+34.620379999 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5e0761f7-70d1-4952-b645-1b99f46e3fc7-metrics-tls") pod "dns-default-g54h7" (UID: "5e0761f7-70d1-4952-b645-1b99f46e3fc7") : secret "dns-default-metrics-tls" not found Sep 09 06:38:56.224046 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.223975 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e0761f7-70d1-4952-b645-1b99f46e3fc7-tmp-dir\") pod \"dns-default-g54h7\" (UID: \"5e0761f7-70d1-4952-b645-1b99f46e3fc7\") " pod="openshift-dns/dns-default-g54h7" Sep 09 06:38:56.224223 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.224119 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5e0761f7-70d1-4952-b645-1b99f46e3fc7-config-volume\") pod \"dns-default-g54h7\" (UID: \"5e0761f7-70d1-4952-b645-1b99f46e3fc7\") " pod="openshift-dns/dns-default-g54h7" Sep 09 06:38:56.226708 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.226681 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7e3c731f-3b9b-445d-a6d9-38c20b30568e-cert\") pod \"ingress-canary-4lksd\" (UID: \"7e3c731f-3b9b-445d-a6d9-38c20b30568e\") " pod="openshift-ingress-canary/ingress-canary-4lksd" Sep 09 06:38:56.231644 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.231620 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcr5s\" (UniqueName: \"kubernetes.io/projected/5e0761f7-70d1-4952-b645-1b99f46e3fc7-kube-api-access-tcr5s\") pod \"dns-default-g54h7\" (UID: \"5e0761f7-70d1-4952-b645-1b99f46e3fc7\") " pod="openshift-dns/dns-default-g54h7" Sep 09 06:38:56.231792 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.231765 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwgdn\" (UniqueName: \"kubernetes.io/projected/7e3c731f-3b9b-445d-a6d9-38c20b30568e-kube-api-access-mwgdn\") pod \"ingress-canary-4lksd\" (UID: \"7e3c731f-3b9b-445d-a6d9-38c20b30568e\") " pod="openshift-ingress-canary/ingress-canary-4lksd" Sep 09 06:38:56.248635 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.248616 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4lksd" Sep 09 06:38:56.324474 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.324442 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhwxb\" (UniqueName: \"kubernetes.io/projected/3355475e-185e-4637-900a-5bf30d29d199-kube-api-access-fhwxb\") pod \"network-check-target-9f6j7\" (UID: \"3355475e-185e-4637-900a-5bf30d29d199\") " pod="openshift-network-diagnostics/network-check-target-9f6j7" Sep 09 06:38:56.324629 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:56.324609 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 06:38:56.324676 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:56.324636 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 06:38:56.324676 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:56.324649 2559 projected.go:196] Error preparing data for projected volume kube-api-access-fhwxb for pod openshift-network-diagnostics/network-check-target-9f6j7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 06:38:56.324780 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:56.324727 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3355475e-185e-4637-900a-5bf30d29d199-kube-api-access-fhwxb podName:3355475e-185e-4637-900a-5bf30d29d199 nodeName:}" failed. No retries permitted until 2026-09-09 06:39:28.324706426 +0000 UTC m=+66.221123828 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-fhwxb" (UniqueName: "kubernetes.io/projected/3355475e-185e-4637-900a-5bf30d29d199-kube-api-access-fhwxb") pod "network-check-target-9f6j7" (UID: "3355475e-185e-4637-900a-5bf30d29d199") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 06:38:56.626069 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.626019 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-tls\") pod \"image-registry-798d7b96c-6srkk\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:38:56.626298 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.626091 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/635a2ebb-6998-49e2-8e4d-088725c2c079-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-5x9qz\" (UID: \"635a2ebb-6998-49e2-8e4d-088725c2c079\") " pod="openshift-network-console/networking-console-plugin-79977f9844-5x9qz" Sep 09 06:38:56.626298 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:56.626188 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 06:38:56.626298 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:56.626212 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-798d7b96c-6srkk: secret "image-registry-tls" not found Sep 09 06:38:56.626298 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:56.626257 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 09 06:38:56.626298 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:56.626268 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-tls podName:fe89f009-e9d3-404a-b39a-895d216f3c22 nodeName:}" failed. No retries permitted until 2026-09-09 06:38:57.626254007 +0000 UTC m=+35.522671407 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-tls") pod "image-registry-798d7b96c-6srkk" (UID: "fe89f009-e9d3-404a-b39a-895d216f3c22") : secret "image-registry-tls" not found Sep 09 06:38:56.626527 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:56.626315 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/635a2ebb-6998-49e2-8e4d-088725c2c079-networking-console-plugin-cert podName:635a2ebb-6998-49e2-8e4d-088725c2c079 nodeName:}" failed. No retries permitted until 2026-09-09 06:38:57.626299132 +0000 UTC m=+35.522716547 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/635a2ebb-6998-49e2-8e4d-088725c2c079-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-5x9qz" (UID: "635a2ebb-6998-49e2-8e4d-088725c2c079") : secret "networking-console-plugin-cert" not found Sep 09 06:38:56.727374 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:56.727337 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e0761f7-70d1-4952-b645-1b99f46e3fc7-metrics-tls\") pod \"dns-default-g54h7\" (UID: \"5e0761f7-70d1-4952-b645-1b99f46e3fc7\") " pod="openshift-dns/dns-default-g54h7" Sep 09 06:38:56.727780 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:56.727473 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 06:38:56.727780 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:56.727537 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e0761f7-70d1-4952-b645-1b99f46e3fc7-metrics-tls podName:5e0761f7-70d1-4952-b645-1b99f46e3fc7 nodeName:}" failed. No retries permitted until 2026-09-09 06:38:57.727517233 +0000 UTC m=+35.623934661 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5e0761f7-70d1-4952-b645-1b99f46e3fc7-metrics-tls") pod "dns-default-g54h7" (UID: "5e0761f7-70d1-4952-b645-1b99f46e3fc7") : secret "dns-default-metrics-tls" not found Sep 09 06:38:57.634740 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:57.634533 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-tls\") pod \"image-registry-798d7b96c-6srkk\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:38:57.634932 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:57.634778 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/635a2ebb-6998-49e2-8e4d-088725c2c079-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-5x9qz\" (UID: \"635a2ebb-6998-49e2-8e4d-088725c2c079\") " pod="openshift-network-console/networking-console-plugin-79977f9844-5x9qz" Sep 09 06:38:57.634932 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:57.634679 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 06:38:57.634932 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:57.634827 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-798d7b96c-6srkk: secret "image-registry-tls" not found Sep 09 06:38:57.634932 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:57.634882 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-tls podName:fe89f009-e9d3-404a-b39a-895d216f3c22 nodeName:}" failed. No retries permitted until 2026-09-09 06:38:59.634866708 +0000 UTC m=+37.531284112 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-tls") pod "image-registry-798d7b96c-6srkk" (UID: "fe89f009-e9d3-404a-b39a-895d216f3c22") : secret "image-registry-tls" not found Sep 09 06:38:57.635139 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:57.634961 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 09 06:38:57.635139 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:57.635012 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/635a2ebb-6998-49e2-8e4d-088725c2c079-networking-console-plugin-cert podName:635a2ebb-6998-49e2-8e4d-088725c2c079 nodeName:}" failed. No retries permitted until 2026-09-09 06:38:59.634999079 +0000 UTC m=+37.531416479 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/635a2ebb-6998-49e2-8e4d-088725c2c079-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-5x9qz" (UID: "635a2ebb-6998-49e2-8e4d-088725c2c079") : secret "networking-console-plugin-cert" not found Sep 09 06:38:57.693426 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:57.693387 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4lksd"] Sep 09 06:38:57.696774 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:57.696752 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv72" Sep 09 06:38:57.696774 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:57.696771 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7rdlw" Sep 09 06:38:57.696916 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:57.696850 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f6j7" Sep 09 06:38:57.699901 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:57.699878 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vx54f\"" Sep 09 06:38:57.700004 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:57.699903 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 09 06:38:57.700004 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:57.699903 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 09 06:38:57.700112 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:57.700102 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 09 06:38:57.700189 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:57.700174 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 09 06:38:57.700256 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:57.700185 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tmq86\"" Sep 09 06:38:57.709097 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:38:57.709078 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e3c731f_3b9b_445d_a6d9_38c20b30568e.slice/crio-a5bbb16d7c32fd468958b287550e76a704f77a83b1672ca01e3b3a8dde68577c WatchSource:0}: Error finding container a5bbb16d7c32fd468958b287550e76a704f77a83b1672ca01e3b3a8dde68577c: Status 404 returned error can't find the container with id a5bbb16d7c32fd468958b287550e76a704f77a83b1672ca01e3b3a8dde68577c Sep 09 06:38:57.735500 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:57.735480 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e0761f7-70d1-4952-b645-1b99f46e3fc7-metrics-tls\") pod \"dns-default-g54h7\" (UID: \"5e0761f7-70d1-4952-b645-1b99f46e3fc7\") " pod="openshift-dns/dns-default-g54h7" Sep 09 06:38:57.735757 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:57.735614 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 06:38:57.735757 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:57.735668 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e0761f7-70d1-4952-b645-1b99f46e3fc7-metrics-tls podName:5e0761f7-70d1-4952-b645-1b99f46e3fc7 nodeName:}" failed. No retries permitted until 2026-09-09 06:38:59.735649447 +0000 UTC m=+37.632066846 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5e0761f7-70d1-4952-b645-1b99f46e3fc7-metrics-tls") pod "dns-default-g54h7" (UID: "5e0761f7-70d1-4952-b645-1b99f46e3fc7") : secret "dns-default-metrics-tls" not found Sep 09 06:38:57.881426 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:57.881394 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4lksd" event={"ID":"7e3c731f-3b9b-445d-a6d9-38c20b30568e","Type":"ContainerStarted","Data":"a5bbb16d7c32fd468958b287550e76a704f77a83b1672ca01e3b3a8dde68577c"} Sep 09 06:38:58.886657 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:58.886624 2559 generic.go:356] "Generic (PLEG): container finished" podID="13c06be2-78fb-4ceb-83ee-8cc6a545b395" containerID="be7d5f59526ac50a8a7e1f48014b5b66220eb38ce7b20c7d81647e1889b36d2f" exitCode=0 Sep 09 06:38:58.887249 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:58.886666 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-27vp7" event={"ID":"13c06be2-78fb-4ceb-83ee-8cc6a545b395","Type":"ContainerDied","Data":"be7d5f59526ac50a8a7e1f48014b5b66220eb38ce7b20c7d81647e1889b36d2f"} Sep 09 06:38:59.650038 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:59.649994 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-tls\") pod \"image-registry-798d7b96c-6srkk\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:38:59.650238 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:59.650071 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/635a2ebb-6998-49e2-8e4d-088725c2c079-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-5x9qz\" (UID: \"635a2ebb-6998-49e2-8e4d-088725c2c079\") " pod="openshift-network-console/networking-console-plugin-79977f9844-5x9qz" Sep 09 06:38:59.650238 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:59.650172 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 06:38:59.650238 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:59.650197 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-798d7b96c-6srkk: secret "image-registry-tls" not found Sep 09 06:38:59.650238 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:59.650220 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 09 06:38:59.650419 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:59.650264 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-tls podName:fe89f009-e9d3-404a-b39a-895d216f3c22 nodeName:}" failed. No retries permitted until 2026-09-09 06:39:03.650243798 +0000 UTC m=+41.546661217 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-tls") pod "image-registry-798d7b96c-6srkk" (UID: "fe89f009-e9d3-404a-b39a-895d216f3c22") : secret "image-registry-tls" not found Sep 09 06:38:59.650419 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:59.650282 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/635a2ebb-6998-49e2-8e4d-088725c2c079-networking-console-plugin-cert podName:635a2ebb-6998-49e2-8e4d-088725c2c079 nodeName:}" failed. No retries permitted until 2026-09-09 06:39:03.650272744 +0000 UTC m=+41.546690146 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/635a2ebb-6998-49e2-8e4d-088725c2c079-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-5x9qz" (UID: "635a2ebb-6998-49e2-8e4d-088725c2c079") : secret "networking-console-plugin-cert" not found Sep 09 06:38:59.750788 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:59.750754 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e0761f7-70d1-4952-b645-1b99f46e3fc7-metrics-tls\") pod \"dns-default-g54h7\" (UID: \"5e0761f7-70d1-4952-b645-1b99f46e3fc7\") " pod="openshift-dns/dns-default-g54h7" Sep 09 06:38:59.750942 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:59.750922 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 06:38:59.750999 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:38:59.750988 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e0761f7-70d1-4952-b645-1b99f46e3fc7-metrics-tls podName:5e0761f7-70d1-4952-b645-1b99f46e3fc7 nodeName:}" failed. No retries permitted until 2026-09-09 06:39:03.75097318 +0000 UTC m=+41.647390581 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5e0761f7-70d1-4952-b645-1b99f46e3fc7-metrics-tls") pod "dns-default-g54h7" (UID: "5e0761f7-70d1-4952-b645-1b99f46e3fc7") : secret "dns-default-metrics-tls" not found Sep 09 06:38:59.894357 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:59.894326 2559 generic.go:356] "Generic (PLEG): container finished" podID="13c06be2-78fb-4ceb-83ee-8cc6a545b395" containerID="0422f5611c289fb99824ef7354f54f4a52c740953e9d9b9127b22961ddc04565" exitCode=0 Sep 09 06:38:59.894796 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:38:59.894371 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-27vp7" event={"ID":"13c06be2-78fb-4ceb-83ee-8cc6a545b395","Type":"ContainerDied","Data":"0422f5611c289fb99824ef7354f54f4a52c740953e9d9b9127b22961ddc04565"} Sep 09 06:39:00.898707 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:39:00.898410 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4lksd" event={"ID":"7e3c731f-3b9b-445d-a6d9-38c20b30568e","Type":"ContainerStarted","Data":"bf1acb1511f1705d74dfe5df4bda9e91d674008505a07f8f65581ba30ba058f8"} Sep 09 06:39:00.902019 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:39:00.901986 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-27vp7" event={"ID":"13c06be2-78fb-4ceb-83ee-8cc6a545b395","Type":"ContainerStarted","Data":"6aaebe47dac2f12d2f60728875f134dcaa3b443e5f7b8ab6c17a828a36f1aec0"} Sep 09 06:39:00.915560 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:39:00.915511 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4lksd" podStartSLOduration=2.999539272 podStartE2EDuration="5.915492702s" totalImagesPullingTime="2.91595343s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:38:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:39:00.915332854 +0000 UTC m=+38.811750276" watchObservedRunningTime="2026-09-09 06:39:00.915492702 +0000 UTC m=+38.811910121" Sep 09 06:39:00.939981 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:39:00.939933 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-27vp7" podStartSLOduration=9.773431472 podStartE2EDuration="38.939919095s" totalImagesPullingTime="29.166487623s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 06:38:22 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:39:00.939184553 +0000 UTC m=+38.835601975" watchObservedRunningTime="2026-09-09 06:39:00.939919095 +0000 UTC m=+38.836336514" Sep 09 06:39:03.684739 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:39:03.684694 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-tls\") pod \"image-registry-798d7b96c-6srkk\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:39:03.685191 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:39:03.684754 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/635a2ebb-6998-49e2-8e4d-088725c2c079-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-5x9qz\" (UID: \"635a2ebb-6998-49e2-8e4d-088725c2c079\") " pod="openshift-network-console/networking-console-plugin-79977f9844-5x9qz" Sep 09 06:39:03.685191 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:39:03.684852 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 06:39:03.685191 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:39:03.684870 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-798d7b96c-6srkk: secret "image-registry-tls" not found Sep 09 06:39:03.685191 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:39:03.684927 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-tls podName:fe89f009-e9d3-404a-b39a-895d216f3c22 nodeName:}" failed. No retries permitted until 2026-09-09 06:39:11.684910609 +0000 UTC m=+49.581328026 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-tls") pod "image-registry-798d7b96c-6srkk" (UID: "fe89f009-e9d3-404a-b39a-895d216f3c22") : secret "image-registry-tls" not found Sep 09 06:39:03.685191 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:39:03.684939 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 09 06:39:03.685191 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:39:03.685004 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/635a2ebb-6998-49e2-8e4d-088725c2c079-networking-console-plugin-cert podName:635a2ebb-6998-49e2-8e4d-088725c2c079 nodeName:}" failed. No retries permitted until 2026-09-09 06:39:11.684984106 +0000 UTC m=+49.581401509 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/635a2ebb-6998-49e2-8e4d-088725c2c079-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-5x9qz" (UID: "635a2ebb-6998-49e2-8e4d-088725c2c079") : secret "networking-console-plugin-cert" not found Sep 09 06:39:03.785897 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:39:03.785867 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e0761f7-70d1-4952-b645-1b99f46e3fc7-metrics-tls\") pod \"dns-default-g54h7\" (UID: \"5e0761f7-70d1-4952-b645-1b99f46e3fc7\") " pod="openshift-dns/dns-default-g54h7" Sep 09 06:39:03.786113 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:39:03.786012 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 06:39:03.786113 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:39:03.786091 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e0761f7-70d1-4952-b645-1b99f46e3fc7-metrics-tls podName:5e0761f7-70d1-4952-b645-1b99f46e3fc7 nodeName:}" failed. No retries permitted until 2026-09-09 06:39:11.7860762 +0000 UTC m=+49.682493614 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5e0761f7-70d1-4952-b645-1b99f46e3fc7-metrics-tls") pod "dns-default-g54h7" (UID: "5e0761f7-70d1-4952-b645-1b99f46e3fc7") : secret "dns-default-metrics-tls" not found Sep 09 06:39:11.737565 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:39:11.737519 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/635a2ebb-6998-49e2-8e4d-088725c2c079-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-5x9qz\" (UID: \"635a2ebb-6998-49e2-8e4d-088725c2c079\") " pod="openshift-network-console/networking-console-plugin-79977f9844-5x9qz" Sep 09 06:39:11.738024 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:39:11.737643 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-tls\") pod \"image-registry-798d7b96c-6srkk\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:39:11.738024 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:39:11.737684 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 09 06:39:11.738024 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:39:11.737730 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 06:39:11.738024 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:39:11.737742 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-798d7b96c-6srkk: secret "image-registry-tls" not found Sep 09 06:39:11.738024 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:39:11.737767 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/635a2ebb-6998-49e2-8e4d-088725c2c079-networking-console-plugin-cert podName:635a2ebb-6998-49e2-8e4d-088725c2c079 nodeName:}" failed. No retries permitted until 2026-09-09 06:39:27.737748533 +0000 UTC m=+65.634165954 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/635a2ebb-6998-49e2-8e4d-088725c2c079-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-5x9qz" (UID: "635a2ebb-6998-49e2-8e4d-088725c2c079") : secret "networking-console-plugin-cert" not found Sep 09 06:39:11.738024 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:39:11.737810 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-tls podName:fe89f009-e9d3-404a-b39a-895d216f3c22 nodeName:}" failed. No retries permitted until 2026-09-09 06:39:27.737798969 +0000 UTC m=+65.634216369 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-tls") pod "image-registry-798d7b96c-6srkk" (UID: "fe89f009-e9d3-404a-b39a-895d216f3c22") : secret "image-registry-tls" not found Sep 09 06:39:11.838264 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:39:11.838238 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e0761f7-70d1-4952-b645-1b99f46e3fc7-metrics-tls\") pod \"dns-default-g54h7\" (UID: \"5e0761f7-70d1-4952-b645-1b99f46e3fc7\") " pod="openshift-dns/dns-default-g54h7" Sep 09 06:39:11.838361 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:39:11.838334 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 06:39:11.838398 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:39:11.838384 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e0761f7-70d1-4952-b645-1b99f46e3fc7-metrics-tls podName:5e0761f7-70d1-4952-b645-1b99f46e3fc7 nodeName:}" failed. No retries permitted until 2026-09-09 06:39:27.838372532 +0000 UTC m=+65.734789932 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5e0761f7-70d1-4952-b645-1b99f46e3fc7-metrics-tls") pod "dns-default-g54h7" (UID: "5e0761f7-70d1-4952-b645-1b99f46e3fc7") : secret "dns-default-metrics-tls" not found Sep 09 06:39:21.871995 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:39:21.871967 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-snmsv" Sep 09 06:39:27.752248 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:39:27.752206 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/635a2ebb-6998-49e2-8e4d-088725c2c079-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-5x9qz\" (UID: \"635a2ebb-6998-49e2-8e4d-088725c2c079\") " pod="openshift-network-console/networking-console-plugin-79977f9844-5x9qz" Sep 09 06:39:27.752627 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:39:27.752298 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-tls\") pod \"image-registry-798d7b96c-6srkk\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:39:27.752627 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:39:27.752364 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 09 06:39:27.752627 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:39:27.752380 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 06:39:27.752627 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:39:27.752390 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-798d7b96c-6srkk: secret "image-registry-tls" not found Sep 09 06:39:27.752627 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:39:27.752431 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/635a2ebb-6998-49e2-8e4d-088725c2c079-networking-console-plugin-cert podName:635a2ebb-6998-49e2-8e4d-088725c2c079 nodeName:}" failed. No retries permitted until 2026-09-09 06:39:59.752415158 +0000 UTC m=+97.648832584 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/635a2ebb-6998-49e2-8e4d-088725c2c079-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-5x9qz" (UID: "635a2ebb-6998-49e2-8e4d-088725c2c079") : secret "networking-console-plugin-cert" not found Sep 09 06:39:27.752627 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:39:27.752447 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-tls podName:fe89f009-e9d3-404a-b39a-895d216f3c22 nodeName:}" failed. No retries permitted until 2026-09-09 06:39:59.752440321 +0000 UTC m=+97.648857721 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-tls") pod "image-registry-798d7b96c-6srkk" (UID: "fe89f009-e9d3-404a-b39a-895d216f3c22") : secret "image-registry-tls" not found Sep 09 06:39:27.853068 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:39:27.853028 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e0761f7-70d1-4952-b645-1b99f46e3fc7-metrics-tls\") pod \"dns-default-g54h7\" (UID: \"5e0761f7-70d1-4952-b645-1b99f46e3fc7\") " pod="openshift-dns/dns-default-g54h7" Sep 09 06:39:27.853243 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:39:27.853170 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 06:39:27.853243 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:39:27.853231 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e0761f7-70d1-4952-b645-1b99f46e3fc7-metrics-tls podName:5e0761f7-70d1-4952-b645-1b99f46e3fc7 nodeName:}" failed. No retries permitted until 2026-09-09 06:39:59.853217226 +0000 UTC m=+97.749634631 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5e0761f7-70d1-4952-b645-1b99f46e3fc7-metrics-tls") pod "dns-default-g54h7" (UID: "5e0761f7-70d1-4952-b645-1b99f46e3fc7") : secret "dns-default-metrics-tls" not found Sep 09 06:39:28.255901 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:39:28.255863 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/11156d42-6fd0-4887-9f80-6f172dcce6c1-original-pull-secret\") pod \"global-pull-secret-syncer-7rdlw\" (UID: \"11156d42-6fd0-4887-9f80-6f172dcce6c1\") " pod="kube-system/global-pull-secret-syncer-7rdlw" Sep 09 06:39:28.255901 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:39:28.255905 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/336552c8-ca0f-434a-8353-bc61ea118fa5-metrics-certs\") pod \"network-metrics-daemon-pkv72\" (UID: \"336552c8-ca0f-434a-8353-bc61ea118fa5\") " pod="openshift-multus/network-metrics-daemon-pkv72" Sep 09 06:39:28.258523 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:39:28.258502 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 09 06:39:28.258569 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:39:28.258524 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 09 06:39:28.266084 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:39:28.266064 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 09 06:39:28.266191 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:39:28.266119 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/336552c8-ca0f-434a-8353-bc61ea118fa5-metrics-certs podName:336552c8-ca0f-434a-8353-bc61ea118fa5 nodeName:}" failed. No retries permitted until 2026-09-09 06:40:32.266103168 +0000 UTC m=+130.162520568 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/336552c8-ca0f-434a-8353-bc61ea118fa5-metrics-certs") pod "network-metrics-daemon-pkv72" (UID: "336552c8-ca0f-434a-8353-bc61ea118fa5") : secret "metrics-daemon-secret" not found Sep 09 06:39:28.269126 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:39:28.269098 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/11156d42-6fd0-4887-9f80-6f172dcce6c1-original-pull-secret\") pod \"global-pull-secret-syncer-7rdlw\" (UID: \"11156d42-6fd0-4887-9f80-6f172dcce6c1\") " pod="kube-system/global-pull-secret-syncer-7rdlw" Sep 09 06:39:28.322258 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:39:28.322237 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7rdlw" Sep 09 06:39:28.357358 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:39:28.357319 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhwxb\" (UniqueName: \"kubernetes.io/projected/3355475e-185e-4637-900a-5bf30d29d199-kube-api-access-fhwxb\") pod \"network-check-target-9f6j7\" (UID: \"3355475e-185e-4637-900a-5bf30d29d199\") " pod="openshift-network-diagnostics/network-check-target-9f6j7" Sep 09 06:39:28.359467 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:39:28.359447 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 09 06:39:28.370325 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:39:28.370251 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 09 06:39:28.380856 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:39:28.380829 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhwxb\" (UniqueName: \"kubernetes.io/projected/3355475e-185e-4637-900a-5bf30d29d199-kube-api-access-fhwxb\") pod \"network-check-target-9f6j7\" (UID: \"3355475e-185e-4637-900a-5bf30d29d199\") " pod="openshift-network-diagnostics/network-check-target-9f6j7" Sep 09 06:39:28.588428 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:39:28.588347 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7rdlw"] Sep 09 06:39:28.591933 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:39:28.591908 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11156d42_6fd0_4887_9f80_6f172dcce6c1.slice/crio-63a4082b51939b0c33fffcd58534e6243624b7530d51e89ccaf9886de98770d4 WatchSource:0}: Error finding container 63a4082b51939b0c33fffcd58534e6243624b7530d51e89ccaf9886de98770d4: Status 404 returned error can't find the container with id 63a4082b51939b0c33fffcd58534e6243624b7530d51e89ccaf9886de98770d4 Sep 09 06:39:28.618828 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:39:28.618808 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vx54f\"" Sep 09 06:39:28.627631 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:39:28.627612 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f6j7" Sep 09 06:39:28.893727 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:39:28.893660 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9f6j7"] Sep 09 06:39:28.897148 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:39:28.897118 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3355475e_185e_4637_900a_5bf30d29d199.slice/crio-53ce65b2a0cdc754c465b0f3b81267f180eed6586b931283a06b989ecd490a1d WatchSource:0}: Error finding container 53ce65b2a0cdc754c465b0f3b81267f180eed6586b931283a06b989ecd490a1d: Status 404 returned error can't find the container with id 53ce65b2a0cdc754c465b0f3b81267f180eed6586b931283a06b989ecd490a1d Sep 09 06:39:28.964755 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:39:28.964707 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9f6j7" event={"ID":"3355475e-185e-4637-900a-5bf30d29d199","Type":"ContainerStarted","Data":"53ce65b2a0cdc754c465b0f3b81267f180eed6586b931283a06b989ecd490a1d"} Sep 09 06:39:28.965690 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:39:28.965669 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7rdlw" event={"ID":"11156d42-6fd0-4887-9f80-6f172dcce6c1","Type":"ContainerStarted","Data":"63a4082b51939b0c33fffcd58534e6243624b7530d51e89ccaf9886de98770d4"} Sep 09 06:39:33.979818 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:39:33.979774 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9f6j7" event={"ID":"3355475e-185e-4637-900a-5bf30d29d199","Type":"ContainerStarted","Data":"a7c17bf27fd91764083eea27f069f9c78139237358ccb77a31ff405580d0f154"} Sep 09 06:39:33.980242 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:39:33.979880 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9f6j7" Sep 09 06:39:33.981187 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:39:33.981165 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7rdlw" event={"ID":"11156d42-6fd0-4887-9f80-6f172dcce6c1","Type":"ContainerStarted","Data":"99c177f8a9cf3969adaea43c2360e0abc11643c851a4d15372141d571feb38a7"} Sep 09 06:39:33.993540 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:39:33.993492 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9f6j7" podStartSLOduration=67.342539982 podStartE2EDuration="1m11.993479333s" totalImagesPullingTime="4.650939351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:38:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:39:33.992719678 +0000 UTC m=+71.889137101" watchObservedRunningTime="2026-09-09 06:39:33.993479333 +0000 UTC m=+71.889896754" Sep 09 06:39:34.009322 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:39:34.009280 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-7rdlw" podStartSLOduration=67.040294242 podStartE2EDuration="1m12.009266324s" totalImagesPullingTime="4.968972082s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 06:38:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:39:34.008826695 +0000 UTC m=+71.905244116" watchObservedRunningTime="2026-09-09 06:39:34.009266324 +0000 UTC m=+71.905683745" Sep 09 06:39:59.772673 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:39:59.772636 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-tls\") pod \"image-registry-798d7b96c-6srkk\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:39:59.773023 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:39:59.772686 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/635a2ebb-6998-49e2-8e4d-088725c2c079-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-5x9qz\" (UID: \"635a2ebb-6998-49e2-8e4d-088725c2c079\") " pod="openshift-network-console/networking-console-plugin-79977f9844-5x9qz" Sep 09 06:39:59.773023 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:39:59.772786 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 09 06:39:59.773023 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:39:59.772792 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 06:39:59.773023 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:39:59.772817 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-798d7b96c-6srkk: secret "image-registry-tls" not found Sep 09 06:39:59.773023 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:39:59.772847 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/635a2ebb-6998-49e2-8e4d-088725c2c079-networking-console-plugin-cert podName:635a2ebb-6998-49e2-8e4d-088725c2c079 nodeName:}" failed. No retries permitted until 2026-09-09 06:41:03.772833374 +0000 UTC m=+161.669250774 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/635a2ebb-6998-49e2-8e4d-088725c2c079-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-5x9qz" (UID: "635a2ebb-6998-49e2-8e4d-088725c2c079") : secret "networking-console-plugin-cert" not found Sep 09 06:39:59.773023 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:39:59.772868 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-tls podName:fe89f009-e9d3-404a-b39a-895d216f3c22 nodeName:}" failed. No retries permitted until 2026-09-09 06:41:03.772855102 +0000 UTC m=+161.669272506 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-tls") pod "image-registry-798d7b96c-6srkk" (UID: "fe89f009-e9d3-404a-b39a-895d216f3c22") : secret "image-registry-tls" not found Sep 09 06:39:59.873081 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:39:59.873026 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e0761f7-70d1-4952-b645-1b99f46e3fc7-metrics-tls\") pod \"dns-default-g54h7\" (UID: \"5e0761f7-70d1-4952-b645-1b99f46e3fc7\") " pod="openshift-dns/dns-default-g54h7" Sep 09 06:39:59.873216 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:39:59.873179 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 06:39:59.873259 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:39:59.873253 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e0761f7-70d1-4952-b645-1b99f46e3fc7-metrics-tls podName:5e0761f7-70d1-4952-b645-1b99f46e3fc7 nodeName:}" failed. No retries permitted until 2026-09-09 06:41:03.873235868 +0000 UTC m=+161.769653282 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5e0761f7-70d1-4952-b645-1b99f46e3fc7-metrics-tls") pod "dns-default-g54h7" (UID: "5e0761f7-70d1-4952-b645-1b99f46e3fc7") : secret "dns-default-metrics-tls" not found Sep 09 06:40:04.986128 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:40:04.986095 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9f6j7" Sep 09 06:40:32.309326 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:40:32.309269 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/336552c8-ca0f-434a-8353-bc61ea118fa5-metrics-certs\") pod \"network-metrics-daemon-pkv72\" (UID: \"336552c8-ca0f-434a-8353-bc61ea118fa5\") " pod="openshift-multus/network-metrics-daemon-pkv72" Sep 09 06:40:32.309847 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:40:32.309394 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 09 06:40:32.309847 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:40:32.309450 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/336552c8-ca0f-434a-8353-bc61ea118fa5-metrics-certs podName:336552c8-ca0f-434a-8353-bc61ea118fa5 nodeName:}" failed. No retries permitted until 2026-09-09 06:42:34.309435335 +0000 UTC m=+252.205852735 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/336552c8-ca0f-434a-8353-bc61ea118fa5-metrics-certs") pod "network-metrics-daemon-pkv72" (UID: "336552c8-ca0f-434a-8353-bc61ea118fa5") : secret "metrics-daemon-secret" not found Sep 09 06:40:58.909279 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:40:58.909234 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-798d7b96c-6srkk" podUID="fe89f009-e9d3-404a-b39a-895d216f3c22" Sep 09 06:40:58.925950 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:40:58.925921 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-79977f9844-5x9qz" podUID="635a2ebb-6998-49e2-8e4d-088725c2c079" Sep 09 06:40:58.968346 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:40:58.968311 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-g54h7" podUID="5e0761f7-70d1-4952-b645-1b99f46e3fc7" Sep 09 06:40:59.182821 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:40:59.182790 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:41:00.706925 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:41:00.706884 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-pkv72" podUID="336552c8-ca0f-434a-8353-bc61ea118fa5" Sep 09 06:41:03.829396 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:41:03.829355 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-tls\") pod \"image-registry-798d7b96c-6srkk\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:41:03.829840 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:41:03.829415 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/635a2ebb-6998-49e2-8e4d-088725c2c079-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-5x9qz\" (UID: \"635a2ebb-6998-49e2-8e4d-088725c2c079\") " pod="openshift-network-console/networking-console-plugin-79977f9844-5x9qz" Sep 09 06:41:03.829840 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:41:03.829520 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 06:41:03.829840 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:41:03.829539 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-798d7b96c-6srkk: secret "image-registry-tls" not found Sep 09 06:41:03.829840 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:41:03.829548 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 09 06:41:03.829840 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:41:03.829609 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-tls podName:fe89f009-e9d3-404a-b39a-895d216f3c22 nodeName:}" failed. No retries permitted until 2026-09-09 06:43:05.829589302 +0000 UTC m=+283.726006737 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-tls") pod "image-registry-798d7b96c-6srkk" (UID: "fe89f009-e9d3-404a-b39a-895d216f3c22") : secret "image-registry-tls" not found Sep 09 06:41:03.829840 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:41:03.829629 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/635a2ebb-6998-49e2-8e4d-088725c2c079-networking-console-plugin-cert podName:635a2ebb-6998-49e2-8e4d-088725c2c079 nodeName:}" failed. No retries permitted until 2026-09-09 06:43:05.829620255 +0000 UTC m=+283.726037673 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/635a2ebb-6998-49e2-8e4d-088725c2c079-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-5x9qz" (UID: "635a2ebb-6998-49e2-8e4d-088725c2c079") : secret "networking-console-plugin-cert" not found Sep 09 06:41:03.930347 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:41:03.930316 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e0761f7-70d1-4952-b645-1b99f46e3fc7-metrics-tls\") pod \"dns-default-g54h7\" (UID: \"5e0761f7-70d1-4952-b645-1b99f46e3fc7\") " pod="openshift-dns/dns-default-g54h7" Sep 09 06:41:03.930504 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:41:03.930479 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 06:41:03.930567 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:41:03.930552 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e0761f7-70d1-4952-b645-1b99f46e3fc7-metrics-tls podName:5e0761f7-70d1-4952-b645-1b99f46e3fc7 nodeName:}" failed. No retries permitted until 2026-09-09 06:43:05.930532153 +0000 UTC m=+283.826949556 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5e0761f7-70d1-4952-b645-1b99f46e3fc7-metrics-tls") pod "dns-default-g54h7" (UID: "5e0761f7-70d1-4952-b645-1b99f46e3fc7") : secret "dns-default-metrics-tls" not found Sep 09 06:41:10.696907 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:41:10.696866 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-5x9qz" Sep 09 06:41:12.698115 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:41:12.698074 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-g54h7" Sep 09 06:41:13.696739 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:41:13.696701 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv72" Sep 09 06:42:34.360593 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:34.360531 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/336552c8-ca0f-434a-8353-bc61ea118fa5-metrics-certs\") pod \"network-metrics-daemon-pkv72\" (UID: \"336552c8-ca0f-434a-8353-bc61ea118fa5\") " pod="openshift-multus/network-metrics-daemon-pkv72" Sep 09 06:42:34.361135 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:42:34.360683 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 09 06:42:34.361135 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:42:34.360745 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/336552c8-ca0f-434a-8353-bc61ea118fa5-metrics-certs podName:336552c8-ca0f-434a-8353-bc61ea118fa5 nodeName:}" failed. No retries permitted until 2026-09-09 06:44:36.360730997 +0000 UTC m=+374.257148400 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/336552c8-ca0f-434a-8353-bc61ea118fa5-metrics-certs") pod "network-metrics-daemon-pkv72" (UID: "336552c8-ca0f-434a-8353-bc61ea118fa5") : secret "metrics-daemon-secret" not found Sep 09 06:42:46.438971 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.438892 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cflhw"] Sep 09 06:42:46.440783 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.440764 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-rq4hm"] Sep 09 06:42:46.440888 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.440870 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cflhw" Sep 09 06:42:46.442292 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.442275 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-rq4hm" Sep 09 06:42:46.445043 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.445020 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Sep 09 06:42:46.445151 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.445116 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 09 06:42:46.445210 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.445156 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 09 06:42:46.445210 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.445167 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-5z47p\"" Sep 09 06:42:46.445942 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.445924 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-vwqt5\"" Sep 09 06:42:46.447344 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.447326 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Sep 09 06:42:46.452790 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.452765 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-rq4hm"] Sep 09 06:42:46.458167 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.458147 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cflhw"] Sep 09 06:42:46.487294 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.487271 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-4dkcv"] Sep 09 06:42:46.489009 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.488994 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-4dkcv" Sep 09 06:42:46.492312 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.492296 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-8rg2l\"" Sep 09 06:42:46.492603 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.492585 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Sep 09 06:42:46.492773 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.492752 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Sep 09 06:42:46.508747 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.508723 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-4dkcv"] Sep 09 06:42:46.533259 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.533236 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-sqtkx"] Sep 09 06:42:46.534997 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.534981 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-sqtkx" Sep 09 06:42:46.537914 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.537893 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-nf9gf"] Sep 09 06:42:46.538208 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.538185 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Sep 09 06:42:46.538367 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.538348 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-7ltbj\"" Sep 09 06:42:46.538623 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.538607 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Sep 09 06:42:46.538678 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.538624 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Sep 09 06:42:46.538880 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.538865 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Sep 09 06:42:46.539639 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.539625 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-lw25n"] Sep 09 06:42:46.539755 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.539741 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-nf9gf" Sep 09 06:42:46.541315 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.541298 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-lw25n" Sep 09 06:42:46.542946 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.542924 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/39fe02da-f881-42af-9eb7-35cc7e1a0c66-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cflhw\" (UID: \"39fe02da-f881-42af-9eb7-35cc7e1a0c66\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cflhw" Sep 09 06:42:46.543042 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.542966 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r5cl\" (UniqueName: \"kubernetes.io/projected/89e331e0-4596-4390-9b74-b4ddce540b8a-kube-api-access-8r5cl\") pod \"volume-data-source-validator-695c895bd6-4dkcv\" (UID: \"89e331e0-4596-4390-9b74-b4ddce540b8a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-4dkcv" Sep 09 06:42:46.543042 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.543036 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/39fe02da-f881-42af-9eb7-35cc7e1a0c66-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-cflhw\" (UID: \"39fe02da-f881-42af-9eb7-35cc7e1a0c66\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cflhw" Sep 09 06:42:46.543169 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.543096 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7tcg\" (UniqueName: \"kubernetes.io/projected/39fe02da-f881-42af-9eb7-35cc7e1a0c66-kube-api-access-g7tcg\") pod \"cluster-monitoring-operator-6cdcc96965-cflhw\" (UID: \"39fe02da-f881-42af-9eb7-35cc7e1a0c66\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cflhw" Sep 09 06:42:46.543169 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.543121 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q28rv\" (UniqueName: \"kubernetes.io/projected/b25571a8-6f7a-4395-8357-3df81376bbff-kube-api-access-q28rv\") pod \"network-check-source-685456655f-rq4hm\" (UID: \"b25571a8-6f7a-4395-8357-3df81376bbff\") " pod="openshift-network-diagnostics/network-check-source-685456655f-rq4hm" Sep 09 06:42:46.545538 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.545515 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 09 06:42:46.547098 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.547080 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 09 06:42:46.547238 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.547212 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Sep 09 06:42:46.547306 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.547126 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Sep 09 06:42:46.547306 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.547161 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-lmtsd\"" Sep 09 06:42:46.547412 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.547358 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Sep 09 06:42:46.547412 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.547174 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Sep 09 06:42:46.547525 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.547508 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Sep 09 06:42:46.547691 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.547676 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Sep 09 06:42:46.547752 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.547738 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-9rhws\"" Sep 09 06:42:46.552749 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.552726 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-sqtkx"] Sep 09 06:42:46.555458 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.555424 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Sep 09 06:42:46.557243 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.557221 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Sep 09 06:42:46.560169 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.560148 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-nf9gf"] Sep 09 06:42:46.562214 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.562195 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-lw25n"] Sep 09 06:42:46.643790 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.643764 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dbf282a5-cce3-489a-93df-44e7b10ef6fd-serving-cert\") pod \"service-ca-operator-859f94c4b-sqtkx\" (UID: \"dbf282a5-cce3-489a-93df-44e7b10ef6fd\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-sqtkx" Sep 09 06:42:46.643985 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.643803 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dbf282a5-cce3-489a-93df-44e7b10ef6fd-config\") pod \"service-ca-operator-859f94c4b-sqtkx\" (UID: \"dbf282a5-cce3-489a-93df-44e7b10ef6fd\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-sqtkx" Sep 09 06:42:46.643985 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.643828 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/39fe02da-f881-42af-9eb7-35cc7e1a0c66-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-cflhw\" (UID: \"39fe02da-f881-42af-9eb7-35cc7e1a0c66\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cflhw" Sep 09 06:42:46.643985 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.643873 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8r5cl\" (UniqueName: \"kubernetes.io/projected/89e331e0-4596-4390-9b74-b4ddce540b8a-kube-api-access-8r5cl\") pod \"volume-data-source-validator-695c895bd6-4dkcv\" (UID: \"89e331e0-4596-4390-9b74-b4ddce540b8a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-4dkcv" Sep 09 06:42:46.644190 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.644022 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/781bd79e-3b6d-493f-a5e1-aaeecb110ffd-tmp\") pod \"insights-operator-76bcd6ccf5-lw25n\" (UID: \"781bd79e-3b6d-493f-a5e1-aaeecb110ffd\") " pod="openshift-insights/insights-operator-76bcd6ccf5-lw25n" Sep 09 06:42:46.644190 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.644077 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/26d28788-4f98-454b-ab39-2d0d857aa674-config\") pod \"console-operator-85b454874b-nf9gf\" (UID: \"26d28788-4f98-454b-ab39-2d0d857aa674\") " pod="openshift-console-operator/console-operator-85b454874b-nf9gf" Sep 09 06:42:46.644190 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.644117 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/781bd79e-3b6d-493f-a5e1-aaeecb110ffd-snapshots\") pod \"insights-operator-76bcd6ccf5-lw25n\" (UID: \"781bd79e-3b6d-493f-a5e1-aaeecb110ffd\") " pod="openshift-insights/insights-operator-76bcd6ccf5-lw25n" Sep 09 06:42:46.644190 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.644146 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/26d28788-4f98-454b-ab39-2d0d857aa674-serving-cert\") pod \"console-operator-85b454874b-nf9gf\" (UID: \"26d28788-4f98-454b-ab39-2d0d857aa674\") " pod="openshift-console-operator/console-operator-85b454874b-nf9gf" Sep 09 06:42:46.644190 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.644182 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgtnr\" (UniqueName: \"kubernetes.io/projected/781bd79e-3b6d-493f-a5e1-aaeecb110ffd-kube-api-access-pgtnr\") pod \"insights-operator-76bcd6ccf5-lw25n\" (UID: \"781bd79e-3b6d-493f-a5e1-aaeecb110ffd\") " pod="openshift-insights/insights-operator-76bcd6ccf5-lw25n" Sep 09 06:42:46.644437 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.644221 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42rh7\" (UniqueName: \"kubernetes.io/projected/dbf282a5-cce3-489a-93df-44e7b10ef6fd-kube-api-access-42rh7\") pod \"service-ca-operator-859f94c4b-sqtkx\" (UID: \"dbf282a5-cce3-489a-93df-44e7b10ef6fd\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-sqtkx" Sep 09 06:42:46.644437 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.644240 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/781bd79e-3b6d-493f-a5e1-aaeecb110ffd-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-lw25n\" (UID: \"781bd79e-3b6d-493f-a5e1-aaeecb110ffd\") " pod="openshift-insights/insights-operator-76bcd6ccf5-lw25n" Sep 09 06:42:46.644437 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.644270 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/781bd79e-3b6d-493f-a5e1-aaeecb110ffd-serving-cert\") pod \"insights-operator-76bcd6ccf5-lw25n\" (UID: \"781bd79e-3b6d-493f-a5e1-aaeecb110ffd\") " pod="openshift-insights/insights-operator-76bcd6ccf5-lw25n" Sep 09 06:42:46.644437 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.644331 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/781bd79e-3b6d-493f-a5e1-aaeecb110ffd-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-lw25n\" (UID: \"781bd79e-3b6d-493f-a5e1-aaeecb110ffd\") " pod="openshift-insights/insights-operator-76bcd6ccf5-lw25n" Sep 09 06:42:46.644437 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.644365 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7tcg\" (UniqueName: \"kubernetes.io/projected/39fe02da-f881-42af-9eb7-35cc7e1a0c66-kube-api-access-g7tcg\") pod \"cluster-monitoring-operator-6cdcc96965-cflhw\" (UID: \"39fe02da-f881-42af-9eb7-35cc7e1a0c66\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cflhw" Sep 09 06:42:46.644437 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.644383 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q28rv\" (UniqueName: \"kubernetes.io/projected/b25571a8-6f7a-4395-8357-3df81376bbff-kube-api-access-q28rv\") pod \"network-check-source-685456655f-rq4hm\" (UID: \"b25571a8-6f7a-4395-8357-3df81376bbff\") " pod="openshift-network-diagnostics/network-check-source-685456655f-rq4hm" Sep 09 06:42:46.644437 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.644398 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/26d28788-4f98-454b-ab39-2d0d857aa674-trusted-ca\") pod \"console-operator-85b454874b-nf9gf\" (UID: \"26d28788-4f98-454b-ab39-2d0d857aa674\") " pod="openshift-console-operator/console-operator-85b454874b-nf9gf" Sep 09 06:42:46.644437 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.644423 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/39fe02da-f881-42af-9eb7-35cc7e1a0c66-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cflhw\" (UID: \"39fe02da-f881-42af-9eb7-35cc7e1a0c66\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cflhw" Sep 09 06:42:46.644437 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.644438 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jk686\" (UniqueName: \"kubernetes.io/projected/26d28788-4f98-454b-ab39-2d0d857aa674-kube-api-access-jk686\") pod \"console-operator-85b454874b-nf9gf\" (UID: \"26d28788-4f98-454b-ab39-2d0d857aa674\") " pod="openshift-console-operator/console-operator-85b454874b-nf9gf" Sep 09 06:42:46.644828 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:42:46.644533 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 09 06:42:46.644828 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:42:46.644593 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39fe02da-f881-42af-9eb7-35cc7e1a0c66-cluster-monitoring-operator-tls podName:39fe02da-f881-42af-9eb7-35cc7e1a0c66 nodeName:}" failed. No retries permitted until 2026-09-09 06:42:47.144576365 +0000 UTC m=+265.040993780 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/39fe02da-f881-42af-9eb7-35cc7e1a0c66-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-cflhw" (UID: "39fe02da-f881-42af-9eb7-35cc7e1a0c66") : secret "cluster-monitoring-operator-tls" not found Sep 09 06:42:46.644828 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.644662 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/39fe02da-f881-42af-9eb7-35cc7e1a0c66-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-cflhw\" (UID: \"39fe02da-f881-42af-9eb7-35cc7e1a0c66\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cflhw" Sep 09 06:42:46.654409 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.654387 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r5cl\" (UniqueName: \"kubernetes.io/projected/89e331e0-4596-4390-9b74-b4ddce540b8a-kube-api-access-8r5cl\") pod \"volume-data-source-validator-695c895bd6-4dkcv\" (UID: \"89e331e0-4596-4390-9b74-b4ddce540b8a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-4dkcv" Sep 09 06:42:46.657966 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.657943 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7tcg\" (UniqueName: \"kubernetes.io/projected/39fe02da-f881-42af-9eb7-35cc7e1a0c66-kube-api-access-g7tcg\") pod \"cluster-monitoring-operator-6cdcc96965-cflhw\" (UID: \"39fe02da-f881-42af-9eb7-35cc7e1a0c66\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cflhw" Sep 09 06:42:46.659365 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.659346 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q28rv\" (UniqueName: \"kubernetes.io/projected/b25571a8-6f7a-4395-8357-3df81376bbff-kube-api-access-q28rv\") pod \"network-check-source-685456655f-rq4hm\" (UID: \"b25571a8-6f7a-4395-8357-3df81376bbff\") " pod="openshift-network-diagnostics/network-check-source-685456655f-rq4hm" Sep 09 06:42:46.744751 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.744699 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dbf282a5-cce3-489a-93df-44e7b10ef6fd-serving-cert\") pod \"service-ca-operator-859f94c4b-sqtkx\" (UID: \"dbf282a5-cce3-489a-93df-44e7b10ef6fd\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-sqtkx" Sep 09 06:42:46.744751 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.744735 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dbf282a5-cce3-489a-93df-44e7b10ef6fd-config\") pod \"service-ca-operator-859f94c4b-sqtkx\" (UID: \"dbf282a5-cce3-489a-93df-44e7b10ef6fd\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-sqtkx" Sep 09 06:42:46.744870 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.744769 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/781bd79e-3b6d-493f-a5e1-aaeecb110ffd-tmp\") pod \"insights-operator-76bcd6ccf5-lw25n\" (UID: \"781bd79e-3b6d-493f-a5e1-aaeecb110ffd\") " pod="openshift-insights/insights-operator-76bcd6ccf5-lw25n" Sep 09 06:42:46.744870 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.744796 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/26d28788-4f98-454b-ab39-2d0d857aa674-config\") pod \"console-operator-85b454874b-nf9gf\" (UID: \"26d28788-4f98-454b-ab39-2d0d857aa674\") " pod="openshift-console-operator/console-operator-85b454874b-nf9gf" Sep 09 06:42:46.744973 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.744932 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/781bd79e-3b6d-493f-a5e1-aaeecb110ffd-snapshots\") pod \"insights-operator-76bcd6ccf5-lw25n\" (UID: \"781bd79e-3b6d-493f-a5e1-aaeecb110ffd\") " pod="openshift-insights/insights-operator-76bcd6ccf5-lw25n" Sep 09 06:42:46.745029 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.744984 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/26d28788-4f98-454b-ab39-2d0d857aa674-serving-cert\") pod \"console-operator-85b454874b-nf9gf\" (UID: \"26d28788-4f98-454b-ab39-2d0d857aa674\") " pod="openshift-console-operator/console-operator-85b454874b-nf9gf" Sep 09 06:42:46.745029 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.745016 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgtnr\" (UniqueName: \"kubernetes.io/projected/781bd79e-3b6d-493f-a5e1-aaeecb110ffd-kube-api-access-pgtnr\") pod \"insights-operator-76bcd6ccf5-lw25n\" (UID: \"781bd79e-3b6d-493f-a5e1-aaeecb110ffd\") " pod="openshift-insights/insights-operator-76bcd6ccf5-lw25n" Sep 09 06:42:46.745175 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.745080 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42rh7\" (UniqueName: \"kubernetes.io/projected/dbf282a5-cce3-489a-93df-44e7b10ef6fd-kube-api-access-42rh7\") pod \"service-ca-operator-859f94c4b-sqtkx\" (UID: \"dbf282a5-cce3-489a-93df-44e7b10ef6fd\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-sqtkx" Sep 09 06:42:46.745175 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.745113 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/781bd79e-3b6d-493f-a5e1-aaeecb110ffd-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-lw25n\" (UID: \"781bd79e-3b6d-493f-a5e1-aaeecb110ffd\") " pod="openshift-insights/insights-operator-76bcd6ccf5-lw25n" Sep 09 06:42:46.745175 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.745136 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/781bd79e-3b6d-493f-a5e1-aaeecb110ffd-serving-cert\") pod \"insights-operator-76bcd6ccf5-lw25n\" (UID: \"781bd79e-3b6d-493f-a5e1-aaeecb110ffd\") " pod="openshift-insights/insights-operator-76bcd6ccf5-lw25n" Sep 09 06:42:46.745307 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.745185 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/781bd79e-3b6d-493f-a5e1-aaeecb110ffd-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-lw25n\" (UID: \"781bd79e-3b6d-493f-a5e1-aaeecb110ffd\") " pod="openshift-insights/insights-operator-76bcd6ccf5-lw25n" Sep 09 06:42:46.745307 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.745216 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/26d28788-4f98-454b-ab39-2d0d857aa674-trusted-ca\") pod \"console-operator-85b454874b-nf9gf\" (UID: \"26d28788-4f98-454b-ab39-2d0d857aa674\") " pod="openshift-console-operator/console-operator-85b454874b-nf9gf" Sep 09 06:42:46.745307 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.745259 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jk686\" (UniqueName: \"kubernetes.io/projected/26d28788-4f98-454b-ab39-2d0d857aa674-kube-api-access-jk686\") pod \"console-operator-85b454874b-nf9gf\" (UID: \"26d28788-4f98-454b-ab39-2d0d857aa674\") " pod="openshift-console-operator/console-operator-85b454874b-nf9gf" Sep 09 06:42:46.745479 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.745393 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dbf282a5-cce3-489a-93df-44e7b10ef6fd-config\") pod \"service-ca-operator-859f94c4b-sqtkx\" (UID: \"dbf282a5-cce3-489a-93df-44e7b10ef6fd\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-sqtkx" Sep 09 06:42:46.745479 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.745405 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/781bd79e-3b6d-493f-a5e1-aaeecb110ffd-tmp\") pod \"insights-operator-76bcd6ccf5-lw25n\" (UID: \"781bd79e-3b6d-493f-a5e1-aaeecb110ffd\") " pod="openshift-insights/insights-operator-76bcd6ccf5-lw25n" Sep 09 06:42:46.745479 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.745463 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/26d28788-4f98-454b-ab39-2d0d857aa674-config\") pod \"console-operator-85b454874b-nf9gf\" (UID: \"26d28788-4f98-454b-ab39-2d0d857aa674\") " pod="openshift-console-operator/console-operator-85b454874b-nf9gf" Sep 09 06:42:46.745843 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.745791 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/781bd79e-3b6d-493f-a5e1-aaeecb110ffd-snapshots\") pod \"insights-operator-76bcd6ccf5-lw25n\" (UID: \"781bd79e-3b6d-493f-a5e1-aaeecb110ffd\") " pod="openshift-insights/insights-operator-76bcd6ccf5-lw25n" Sep 09 06:42:46.745950 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.745932 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/781bd79e-3b6d-493f-a5e1-aaeecb110ffd-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-lw25n\" (UID: \"781bd79e-3b6d-493f-a5e1-aaeecb110ffd\") " pod="openshift-insights/insights-operator-76bcd6ccf5-lw25n" Sep 09 06:42:46.746294 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.746272 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/781bd79e-3b6d-493f-a5e1-aaeecb110ffd-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-lw25n\" (UID: \"781bd79e-3b6d-493f-a5e1-aaeecb110ffd\") " pod="openshift-insights/insights-operator-76bcd6ccf5-lw25n" Sep 09 06:42:46.746504 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.746481 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/26d28788-4f98-454b-ab39-2d0d857aa674-trusted-ca\") pod \"console-operator-85b454874b-nf9gf\" (UID: \"26d28788-4f98-454b-ab39-2d0d857aa674\") " pod="openshift-console-operator/console-operator-85b454874b-nf9gf" Sep 09 06:42:46.747378 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.747353 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dbf282a5-cce3-489a-93df-44e7b10ef6fd-serving-cert\") pod \"service-ca-operator-859f94c4b-sqtkx\" (UID: \"dbf282a5-cce3-489a-93df-44e7b10ef6fd\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-sqtkx" Sep 09 06:42:46.747673 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.747655 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/781bd79e-3b6d-493f-a5e1-aaeecb110ffd-serving-cert\") pod \"insights-operator-76bcd6ccf5-lw25n\" (UID: \"781bd79e-3b6d-493f-a5e1-aaeecb110ffd\") " pod="openshift-insights/insights-operator-76bcd6ccf5-lw25n" Sep 09 06:42:46.747860 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.747834 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/26d28788-4f98-454b-ab39-2d0d857aa674-serving-cert\") pod \"console-operator-85b454874b-nf9gf\" (UID: \"26d28788-4f98-454b-ab39-2d0d857aa674\") " pod="openshift-console-operator/console-operator-85b454874b-nf9gf" Sep 09 06:42:46.755322 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.755301 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-rq4hm" Sep 09 06:42:46.756484 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.756459 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jk686\" (UniqueName: \"kubernetes.io/projected/26d28788-4f98-454b-ab39-2d0d857aa674-kube-api-access-jk686\") pod \"console-operator-85b454874b-nf9gf\" (UID: \"26d28788-4f98-454b-ab39-2d0d857aa674\") " pod="openshift-console-operator/console-operator-85b454874b-nf9gf" Sep 09 06:42:46.756736 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.756715 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgtnr\" (UniqueName: \"kubernetes.io/projected/781bd79e-3b6d-493f-a5e1-aaeecb110ffd-kube-api-access-pgtnr\") pod \"insights-operator-76bcd6ccf5-lw25n\" (UID: \"781bd79e-3b6d-493f-a5e1-aaeecb110ffd\") " pod="openshift-insights/insights-operator-76bcd6ccf5-lw25n" Sep 09 06:42:46.757129 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.757110 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42rh7\" (UniqueName: \"kubernetes.io/projected/dbf282a5-cce3-489a-93df-44e7b10ef6fd-kube-api-access-42rh7\") pod \"service-ca-operator-859f94c4b-sqtkx\" (UID: \"dbf282a5-cce3-489a-93df-44e7b10ef6fd\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-sqtkx" Sep 09 06:42:46.796991 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.796704 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-4dkcv" Sep 09 06:42:46.846093 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.845834 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-sqtkx" Sep 09 06:42:46.853530 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.853420 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-nf9gf" Sep 09 06:42:46.858564 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:46.858540 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-lw25n" Sep 09 06:42:47.022313 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:47.022234 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-rq4hm"] Sep 09 06:42:47.025798 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:42:47.025773 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb25571a8_6f7a_4395_8357_3df81376bbff.slice/crio-7793da2a4e5c729a3d7e1e168e104a5d4a2cf070ca02569bfdc9eab30e53a50d WatchSource:0}: Error finding container 7793da2a4e5c729a3d7e1e168e104a5d4a2cf070ca02569bfdc9eab30e53a50d: Status 404 returned error can't find the container with id 7793da2a4e5c729a3d7e1e168e104a5d4a2cf070ca02569bfdc9eab30e53a50d Sep 09 06:42:47.067176 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:47.067146 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-4dkcv"] Sep 09 06:42:47.070297 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:42:47.070275 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89e331e0_4596_4390_9b74_b4ddce540b8a.slice/crio-3dce7639be439cb28733264473e688c74dc4f1eab591c96c6239de299c434ecd WatchSource:0}: Error finding container 3dce7639be439cb28733264473e688c74dc4f1eab591c96c6239de299c434ecd: Status 404 returned error can't find the container with id 3dce7639be439cb28733264473e688c74dc4f1eab591c96c6239de299c434ecd Sep 09 06:42:47.120670 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:47.120634 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-sqtkx"] Sep 09 06:42:47.124707 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:42:47.124669 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbf282a5_cce3_489a_93df_44e7b10ef6fd.slice/crio-88e617af4a96f0d64f3553861a223ff310e7adb4a05da4cc2929bd1d9f4c1205 WatchSource:0}: Error finding container 88e617af4a96f0d64f3553861a223ff310e7adb4a05da4cc2929bd1d9f4c1205: Status 404 returned error can't find the container with id 88e617af4a96f0d64f3553861a223ff310e7adb4a05da4cc2929bd1d9f4c1205 Sep 09 06:42:47.148259 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:47.148231 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/39fe02da-f881-42af-9eb7-35cc7e1a0c66-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cflhw\" (UID: \"39fe02da-f881-42af-9eb7-35cc7e1a0c66\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cflhw" Sep 09 06:42:47.148399 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:42:47.148383 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 09 06:42:47.148452 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:42:47.148444 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39fe02da-f881-42af-9eb7-35cc7e1a0c66-cluster-monitoring-operator-tls podName:39fe02da-f881-42af-9eb7-35cc7e1a0c66 nodeName:}" failed. No retries permitted until 2026-09-09 06:42:48.148426235 +0000 UTC m=+266.044843651 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/39fe02da-f881-42af-9eb7-35cc7e1a0c66-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-cflhw" (UID: "39fe02da-f881-42af-9eb7-35cc7e1a0c66") : secret "cluster-monitoring-operator-tls" not found Sep 09 06:42:47.149966 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:47.149929 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-nf9gf"] Sep 09 06:42:47.153608 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:47.153585 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-lw25n"] Sep 09 06:42:47.154226 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:42:47.154188 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26d28788_4f98_454b_ab39_2d0d857aa674.slice/crio-dd31d9821bcaab3b7390dbbd6e78f7f3e4799000974d08d9feafa6507df51078 WatchSource:0}: Error finding container dd31d9821bcaab3b7390dbbd6e78f7f3e4799000974d08d9feafa6507df51078: Status 404 returned error can't find the container with id dd31d9821bcaab3b7390dbbd6e78f7f3e4799000974d08d9feafa6507df51078 Sep 09 06:42:47.158249 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:42:47.158227 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod781bd79e_3b6d_493f_a5e1_aaeecb110ffd.slice/crio-a366e05572406ba81cf2b6475618565be6f1a98c1fb38c16127e33642a3d1550 WatchSource:0}: Error finding container a366e05572406ba81cf2b6475618565be6f1a98c1fb38c16127e33642a3d1550: Status 404 returned error can't find the container with id a366e05572406ba81cf2b6475618565be6f1a98c1fb38c16127e33642a3d1550 Sep 09 06:42:47.432491 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:47.432460 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-rq4hm" event={"ID":"b25571a8-6f7a-4395-8357-3df81376bbff","Type":"ContainerStarted","Data":"84d6dc338729526bbe89057b78de9b326307d73db2c6ec122bb93533933e63ff"} Sep 09 06:42:47.432655 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:47.432542 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-rq4hm" event={"ID":"b25571a8-6f7a-4395-8357-3df81376bbff","Type":"ContainerStarted","Data":"7793da2a4e5c729a3d7e1e168e104a5d4a2cf070ca02569bfdc9eab30e53a50d"} Sep 09 06:42:47.433610 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:47.433584 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-lw25n" event={"ID":"781bd79e-3b6d-493f-a5e1-aaeecb110ffd","Type":"ContainerStarted","Data":"a366e05572406ba81cf2b6475618565be6f1a98c1fb38c16127e33642a3d1550"} Sep 09 06:42:47.434643 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:47.434618 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-nf9gf" event={"ID":"26d28788-4f98-454b-ab39-2d0d857aa674","Type":"ContainerStarted","Data":"dd31d9821bcaab3b7390dbbd6e78f7f3e4799000974d08d9feafa6507df51078"} Sep 09 06:42:47.435682 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:47.435662 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-sqtkx" event={"ID":"dbf282a5-cce3-489a-93df-44e7b10ef6fd","Type":"ContainerStarted","Data":"88e617af4a96f0d64f3553861a223ff310e7adb4a05da4cc2929bd1d9f4c1205"} Sep 09 06:42:47.436709 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:47.436689 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-4dkcv" event={"ID":"89e331e0-4596-4390-9b74-b4ddce540b8a","Type":"ContainerStarted","Data":"3dce7639be439cb28733264473e688c74dc4f1eab591c96c6239de299c434ecd"} Sep 09 06:42:47.457404 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:47.457361 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-rq4hm" podStartSLOduration=1.4573481830000001 podStartE2EDuration="1.457348183s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:42:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:42:47.456186358 +0000 UTC m=+265.352603783" watchObservedRunningTime="2026-09-09 06:42:47.457348183 +0000 UTC m=+265.353765604" Sep 09 06:42:48.157983 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:48.157315 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/39fe02da-f881-42af-9eb7-35cc7e1a0c66-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cflhw\" (UID: \"39fe02da-f881-42af-9eb7-35cc7e1a0c66\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cflhw" Sep 09 06:42:48.157983 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:42:48.157549 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 09 06:42:48.157983 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:42:48.157616 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39fe02da-f881-42af-9eb7-35cc7e1a0c66-cluster-monitoring-operator-tls podName:39fe02da-f881-42af-9eb7-35cc7e1a0c66 nodeName:}" failed. No retries permitted until 2026-09-09 06:42:50.15759639 +0000 UTC m=+268.054013813 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/39fe02da-f881-42af-9eb7-35cc7e1a0c66-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-cflhw" (UID: "39fe02da-f881-42af-9eb7-35cc7e1a0c66") : secret "cluster-monitoring-operator-tls" not found Sep 09 06:42:49.445127 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:49.445085 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-4dkcv" event={"ID":"89e331e0-4596-4390-9b74-b4ddce540b8a","Type":"ContainerStarted","Data":"d25ce319081300af50f0ee6631db2147b48bd731427948df8fe767cdfdb9b3fa"} Sep 09 06:42:49.461399 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:49.461341 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-4dkcv" podStartSLOduration=2.019066533 podStartE2EDuration="3.461321898s" totalImagesPullingTime="1.442255365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:42:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:42:49.459879285 +0000 UTC m=+267.356296706" watchObservedRunningTime="2026-09-09 06:42:49.461321898 +0000 UTC m=+267.357739319" Sep 09 06:42:49.725516 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:49.725439 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-r456s"] Sep 09 06:42:49.728278 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:49.728203 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-r456s" Sep 09 06:42:49.731393 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:49.731366 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-4kvmr\"" Sep 09 06:42:49.732148 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:49.732118 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Sep 09 06:42:49.732148 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:49.732141 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Sep 09 06:42:49.738532 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:49.738466 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-r456s"] Sep 09 06:42:49.875168 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:49.875135 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dk4wf\" (UniqueName: \"kubernetes.io/projected/8bb61eca-5639-4fd6-bc8e-9807a6ac290f-kube-api-access-dk4wf\") pod \"migrator-fdbb8fdff-r456s\" (UID: \"8bb61eca-5639-4fd6-bc8e-9807a6ac290f\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-r456s" Sep 09 06:42:49.976104 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:49.976000 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dk4wf\" (UniqueName: \"kubernetes.io/projected/8bb61eca-5639-4fd6-bc8e-9807a6ac290f-kube-api-access-dk4wf\") pod \"migrator-fdbb8fdff-r456s\" (UID: \"8bb61eca-5639-4fd6-bc8e-9807a6ac290f\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-r456s" Sep 09 06:42:49.985168 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:49.985142 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dk4wf\" (UniqueName: \"kubernetes.io/projected/8bb61eca-5639-4fd6-bc8e-9807a6ac290f-kube-api-access-dk4wf\") pod \"migrator-fdbb8fdff-r456s\" (UID: \"8bb61eca-5639-4fd6-bc8e-9807a6ac290f\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-r456s" Sep 09 06:42:50.039998 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:50.039960 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-r456s" Sep 09 06:42:50.177146 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:50.177113 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/39fe02da-f881-42af-9eb7-35cc7e1a0c66-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cflhw\" (UID: \"39fe02da-f881-42af-9eb7-35cc7e1a0c66\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cflhw" Sep 09 06:42:50.177269 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:42:50.177252 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 09 06:42:50.177323 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:42:50.177314 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39fe02da-f881-42af-9eb7-35cc7e1a0c66-cluster-monitoring-operator-tls podName:39fe02da-f881-42af-9eb7-35cc7e1a0c66 nodeName:}" failed. No retries permitted until 2026-09-09 06:42:54.177297177 +0000 UTC m=+272.073714577 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/39fe02da-f881-42af-9eb7-35cc7e1a0c66-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-cflhw" (UID: "39fe02da-f881-42af-9eb7-35cc7e1a0c66") : secret "cluster-monitoring-operator-tls" not found Sep 09 06:42:50.455995 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:50.455548 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-r456s"] Sep 09 06:42:50.456744 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:50.456717 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-lw25n" event={"ID":"781bd79e-3b6d-493f-a5e1-aaeecb110ffd","Type":"ContainerStarted","Data":"481dc43b5eb08cfda7e716390e949c5d9091155a28a6762fdb3cbca38f23d669"} Sep 09 06:42:50.458153 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:50.458129 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nf9gf_26d28788-4f98-454b-ab39-2d0d857aa674/console-operator/0.log" Sep 09 06:42:50.458284 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:50.458168 2559 generic.go:356] "Generic (PLEG): container finished" podID="26d28788-4f98-454b-ab39-2d0d857aa674" containerID="b98b59131f5fe598feacb579e477ca414e7049cf5b8a5994487acd99e761b8dc" exitCode=255 Sep 09 06:42:50.458284 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:50.458245 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-nf9gf" event={"ID":"26d28788-4f98-454b-ab39-2d0d857aa674","Type":"ContainerDied","Data":"b98b59131f5fe598feacb579e477ca414e7049cf5b8a5994487acd99e761b8dc"} Sep 09 06:42:50.458529 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:50.458504 2559 scope.go:117] "RemoveContainer" containerID="b98b59131f5fe598feacb579e477ca414e7049cf5b8a5994487acd99e761b8dc" Sep 09 06:42:50.458658 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:42:50.458639 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bb61eca_5639_4fd6_bc8e_9807a6ac290f.slice/crio-e34adf574fe57ce68c6f4a6cda6d53833a9b5266c22261038961a472888432af WatchSource:0}: Error finding container e34adf574fe57ce68c6f4a6cda6d53833a9b5266c22261038961a472888432af: Status 404 returned error can't find the container with id e34adf574fe57ce68c6f4a6cda6d53833a9b5266c22261038961a472888432af Sep 09 06:42:50.460319 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:50.460290 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-sqtkx" event={"ID":"dbf282a5-cce3-489a-93df-44e7b10ef6fd","Type":"ContainerStarted","Data":"fc91b30b55c53f6083eed212946de3de2f4fea186c7f2ea261acff805aba15cb"} Sep 09 06:42:50.475675 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:50.475630 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-lw25n" podStartSLOduration=1.450190721 podStartE2EDuration="4.475618951s" totalImagesPullingTime="3.02542823s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:42:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:42:50.474770201 +0000 UTC m=+268.371187624" watchObservedRunningTime="2026-09-09 06:42:50.475618951 +0000 UTC m=+268.372036367" Sep 09 06:42:50.495538 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:50.495485 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-sqtkx" podStartSLOduration=1.432460334 podStartE2EDuration="4.495465096s" totalImagesPullingTime="3.063004762s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:42:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:42:50.494636142 +0000 UTC m=+268.391053565" watchObservedRunningTime="2026-09-09 06:42:50.495465096 +0000 UTC m=+268.391882523" Sep 09 06:42:51.465904 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:51.465880 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nf9gf_26d28788-4f98-454b-ab39-2d0d857aa674/console-operator/1.log" Sep 09 06:42:51.466349 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:51.466331 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nf9gf_26d28788-4f98-454b-ab39-2d0d857aa674/console-operator/0.log" Sep 09 06:42:51.466410 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:51.466370 2559 generic.go:356] "Generic (PLEG): container finished" podID="26d28788-4f98-454b-ab39-2d0d857aa674" containerID="430b703e1122185a9834704455c8a7e5cc5d4a8b8c0f62e8ab37c85912cc43b9" exitCode=255 Sep 09 06:42:51.466452 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:51.466403 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-nf9gf" event={"ID":"26d28788-4f98-454b-ab39-2d0d857aa674","Type":"ContainerDied","Data":"430b703e1122185a9834704455c8a7e5cc5d4a8b8c0f62e8ab37c85912cc43b9"} Sep 09 06:42:51.466503 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:51.466485 2559 scope.go:117] "RemoveContainer" containerID="b98b59131f5fe598feacb579e477ca414e7049cf5b8a5994487acd99e761b8dc" Sep 09 06:42:51.466687 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:51.466668 2559 scope.go:117] "RemoveContainer" containerID="430b703e1122185a9834704455c8a7e5cc5d4a8b8c0f62e8ab37c85912cc43b9" Sep 09 06:42:51.466877 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:42:51.466851 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-nf9gf_openshift-console-operator(26d28788-4f98-454b-ab39-2d0d857aa674)\"" pod="openshift-console-operator/console-operator-85b454874b-nf9gf" podUID="26d28788-4f98-454b-ab39-2d0d857aa674" Sep 09 06:42:51.467890 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:51.467756 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-r456s" event={"ID":"8bb61eca-5639-4fd6-bc8e-9807a6ac290f","Type":"ContainerStarted","Data":"e34adf574fe57ce68c6f4a6cda6d53833a9b5266c22261038961a472888432af"} Sep 09 06:42:51.870075 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:51.869993 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-s4mlq"] Sep 09 06:42:51.872420 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:51.872399 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-s4mlq" Sep 09 06:42:51.877717 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:51.877690 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ph7nl\"" Sep 09 06:42:51.877832 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:51.877705 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 09 06:42:51.877832 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:51.877725 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 09 06:42:51.924329 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:51.924286 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-s4mlq"] Sep 09 06:42:51.995607 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:51.995282 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-crio-socket\") pod \"insights-runtime-extractor-s4mlq\" (UID: \"93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9\") " pod="openshift-insights/insights-runtime-extractor-s4mlq" Sep 09 06:42:51.995607 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:51.995321 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-data-volume\") pod \"insights-runtime-extractor-s4mlq\" (UID: \"93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9\") " pod="openshift-insights/insights-runtime-extractor-s4mlq" Sep 09 06:42:51.995607 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:51.995345 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gk6h\" (UniqueName: \"kubernetes.io/projected/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-kube-api-access-4gk6h\") pod \"insights-runtime-extractor-s4mlq\" (UID: \"93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9\") " pod="openshift-insights/insights-runtime-extractor-s4mlq" Sep 09 06:42:51.995607 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:51.995407 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s4mlq\" (UID: \"93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9\") " pod="openshift-insights/insights-runtime-extractor-s4mlq" Sep 09 06:42:51.995607 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:51.995508 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s4mlq\" (UID: \"93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9\") " pod="openshift-insights/insights-runtime-extractor-s4mlq" Sep 09 06:42:52.096593 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:52.096568 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s4mlq\" (UID: \"93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9\") " pod="openshift-insights/insights-runtime-extractor-s4mlq" Sep 09 06:42:52.096685 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:52.096607 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-crio-socket\") pod \"insights-runtime-extractor-s4mlq\" (UID: \"93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9\") " pod="openshift-insights/insights-runtime-extractor-s4mlq" Sep 09 06:42:52.096685 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:52.096623 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-data-volume\") pod \"insights-runtime-extractor-s4mlq\" (UID: \"93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9\") " pod="openshift-insights/insights-runtime-extractor-s4mlq" Sep 09 06:42:52.096764 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:52.096696 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-crio-socket\") pod \"insights-runtime-extractor-s4mlq\" (UID: \"93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9\") " pod="openshift-insights/insights-runtime-extractor-s4mlq" Sep 09 06:42:52.096764 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:52.096721 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gk6h\" (UniqueName: \"kubernetes.io/projected/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-kube-api-access-4gk6h\") pod \"insights-runtime-extractor-s4mlq\" (UID: \"93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9\") " pod="openshift-insights/insights-runtime-extractor-s4mlq" Sep 09 06:42:52.096840 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:42:52.096702 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 06:42:52.096840 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:42:52.096813 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 06:42:52.096840 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:52.096833 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-data-volume\") pod \"insights-runtime-extractor-s4mlq\" (UID: \"93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9\") " pod="openshift-insights/insights-runtime-extractor-s4mlq" Sep 09 06:42:52.096935 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:42:52.096839 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-insights-runtime-extractor-tls podName:93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9 nodeName:}" failed. No retries permitted until 2026-09-09 06:42:52.596818546 +0000 UTC m=+270.493235964 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s4mlq" (UID: "93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9") : secret "insights-runtime-extractor-tls" not found Sep 09 06:42:52.096935 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:52.096781 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s4mlq\" (UID: \"93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9\") " pod="openshift-insights/insights-runtime-extractor-s4mlq" Sep 09 06:42:52.096935 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:42:52.096861 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-kube-rbac-proxy-cm podName:93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9 nodeName:}" failed. No retries permitted until 2026-09-09 06:42:52.596850423 +0000 UTC m=+270.493267824 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s4mlq" (UID: "93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9") : configmap "kube-rbac-proxy" not found Sep 09 06:42:52.106701 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:52.106679 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gk6h\" (UniqueName: \"kubernetes.io/projected/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-kube-api-access-4gk6h\") pod \"insights-runtime-extractor-s4mlq\" (UID: \"93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9\") " pod="openshift-insights/insights-runtime-extractor-s4mlq" Sep 09 06:42:52.472626 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:52.472598 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nf9gf_26d28788-4f98-454b-ab39-2d0d857aa674/console-operator/1.log" Sep 09 06:42:52.473027 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:52.472942 2559 scope.go:117] "RemoveContainer" containerID="430b703e1122185a9834704455c8a7e5cc5d4a8b8c0f62e8ab37c85912cc43b9" Sep 09 06:42:52.473155 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:42:52.473130 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-nf9gf_openshift-console-operator(26d28788-4f98-454b-ab39-2d0d857aa674)\"" pod="openshift-console-operator/console-operator-85b454874b-nf9gf" podUID="26d28788-4f98-454b-ab39-2d0d857aa674" Sep 09 06:42:52.474380 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:52.474356 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-r456s" event={"ID":"8bb61eca-5639-4fd6-bc8e-9807a6ac290f","Type":"ContainerStarted","Data":"e4b885900001b6494a9d53cd76125369d54e169c777e9348dc37be8365634bd3"} Sep 09 06:42:52.474509 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:52.474397 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-r456s" event={"ID":"8bb61eca-5639-4fd6-bc8e-9807a6ac290f","Type":"ContainerStarted","Data":"cf3fcc84f549f6764a783385b04fbb14d3183d5b1ed79e541f01820882b3335d"} Sep 09 06:42:52.516377 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:52.516326 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-r456s" podStartSLOduration=2.009693311 podStartE2EDuration="3.516312245s" totalImagesPullingTime="1.506618934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:42:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:42:52.515811255 +0000 UTC m=+270.412228692" watchObservedRunningTime="2026-09-09 06:42:52.516312245 +0000 UTC m=+270.412729667" Sep 09 06:42:52.601065 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:52.601013 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s4mlq\" (UID: \"93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9\") " pod="openshift-insights/insights-runtime-extractor-s4mlq" Sep 09 06:42:52.601225 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:42:52.601154 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 06:42:52.601225 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:52.601209 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s4mlq\" (UID: \"93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9\") " pod="openshift-insights/insights-runtime-extractor-s4mlq" Sep 09 06:42:52.601225 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:42:52.601215 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-kube-rbac-proxy-cm podName:93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9 nodeName:}" failed. No retries permitted until 2026-09-09 06:42:53.601197825 +0000 UTC m=+271.497615225 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s4mlq" (UID: "93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9") : configmap "kube-rbac-proxy" not found Sep 09 06:42:52.601394 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:42:52.601308 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 06:42:52.601394 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:42:52.601377 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-insights-runtime-extractor-tls podName:93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9 nodeName:}" failed. No retries permitted until 2026-09-09 06:42:53.601359083 +0000 UTC m=+271.497776495 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s4mlq" (UID: "93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9") : secret "insights-runtime-extractor-tls" not found Sep 09 06:42:53.608991 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:53.608957 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s4mlq\" (UID: \"93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9\") " pod="openshift-insights/insights-runtime-extractor-s4mlq" Sep 09 06:42:53.609373 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:53.609036 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s4mlq\" (UID: \"93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9\") " pod="openshift-insights/insights-runtime-extractor-s4mlq" Sep 09 06:42:53.609373 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:42:53.609118 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 06:42:53.609373 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:42:53.609136 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 06:42:53.609373 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:42:53.609176 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-kube-rbac-proxy-cm podName:93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9 nodeName:}" failed. No retries permitted until 2026-09-09 06:42:55.609161877 +0000 UTC m=+273.505579277 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s4mlq" (UID: "93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9") : configmap "kube-rbac-proxy" not found Sep 09 06:42:53.609373 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:42:53.609194 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-insights-runtime-extractor-tls podName:93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9 nodeName:}" failed. No retries permitted until 2026-09-09 06:42:55.60918566 +0000 UTC m=+273.505603059 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s4mlq" (UID: "93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9") : secret "insights-runtime-extractor-tls" not found Sep 09 06:42:54.213358 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:54.213317 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/39fe02da-f881-42af-9eb7-35cc7e1a0c66-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cflhw\" (UID: \"39fe02da-f881-42af-9eb7-35cc7e1a0c66\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cflhw" Sep 09 06:42:54.213530 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:42:54.213473 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 09 06:42:54.213580 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:42:54.213542 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39fe02da-f881-42af-9eb7-35cc7e1a0c66-cluster-monitoring-operator-tls podName:39fe02da-f881-42af-9eb7-35cc7e1a0c66 nodeName:}" failed. No retries permitted until 2026-09-09 06:43:02.213527121 +0000 UTC m=+280.109944526 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/39fe02da-f881-42af-9eb7-35cc7e1a0c66-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-cflhw" (UID: "39fe02da-f881-42af-9eb7-35cc7e1a0c66") : secret "cluster-monitoring-operator-tls" not found Sep 09 06:42:54.792240 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:54.792213 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bk8ww_0f25c33f-2f5d-4645-990b-b9d2f201e1fc/dns-node-resolver/0.log" Sep 09 06:42:55.625761 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:55.625726 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s4mlq\" (UID: \"93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9\") " pod="openshift-insights/insights-runtime-extractor-s4mlq" Sep 09 06:42:55.625932 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:55.625789 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s4mlq\" (UID: \"93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9\") " pod="openshift-insights/insights-runtime-extractor-s4mlq" Sep 09 06:42:55.625932 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:42:55.625871 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 06:42:55.625932 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:42:55.625876 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 06:42:55.625932 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:42:55.625924 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-kube-rbac-proxy-cm podName:93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9 nodeName:}" failed. No retries permitted until 2026-09-09 06:42:59.625911063 +0000 UTC m=+277.522328464 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s4mlq" (UID: "93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9") : configmap "kube-rbac-proxy" not found Sep 09 06:42:55.626084 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:42:55.625938 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-insights-runtime-extractor-tls podName:93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9 nodeName:}" failed. No retries permitted until 2026-09-09 06:42:59.625931966 +0000 UTC m=+277.522349365 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s4mlq" (UID: "93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9") : secret "insights-runtime-extractor-tls" not found Sep 09 06:42:55.793442 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:55.793413 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qr5vp_f1a72723-5d93-402c-8ec3-768531d2a883/node-ca/0.log" Sep 09 06:42:56.200423 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:56.200393 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4lksd_7e3c731f-3b9b-445d-a6d9-38c20b30568e/serve-healthcheck-canary/0.log" Sep 09 06:42:56.792922 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:56.792891 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-r456s_8bb61eca-5639-4fd6-bc8e-9807a6ac290f/migrator/0.log" Sep 09 06:42:56.854486 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:56.854453 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-nf9gf" Sep 09 06:42:56.854828 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:56.854538 2559 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-nf9gf" Sep 09 06:42:56.854893 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:56.854877 2559 scope.go:117] "RemoveContainer" containerID="430b703e1122185a9834704455c8a7e5cc5d4a8b8c0f62e8ab37c85912cc43b9" Sep 09 06:42:56.855115 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:42:56.855092 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-nf9gf_openshift-console-operator(26d28788-4f98-454b-ab39-2d0d857aa674)\"" pod="openshift-console-operator/console-operator-85b454874b-nf9gf" podUID="26d28788-4f98-454b-ab39-2d0d857aa674" Sep 09 06:42:56.992423 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:56.992396 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-r456s_8bb61eca-5639-4fd6-bc8e-9807a6ac290f/graceful-termination/0.log" Sep 09 06:42:57.491134 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:57.491106 2559 scope.go:117] "RemoveContainer" containerID="430b703e1122185a9834704455c8a7e5cc5d4a8b8c0f62e8ab37c85912cc43b9" Sep 09 06:42:57.491298 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:42:57.491253 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-nf9gf_openshift-console-operator(26d28788-4f98-454b-ab39-2d0d857aa674)\"" pod="openshift-console-operator/console-operator-85b454874b-nf9gf" podUID="26d28788-4f98-454b-ab39-2d0d857aa674" Sep 09 06:42:59.657034 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:59.656984 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s4mlq\" (UID: \"93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9\") " pod="openshift-insights/insights-runtime-extractor-s4mlq" Sep 09 06:42:59.657440 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:42:59.657125 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 06:42:59.657440 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:42:59.657206 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 06:42:59.657440 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:42:59.657221 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-kube-rbac-proxy-cm podName:93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9 nodeName:}" failed. No retries permitted until 2026-09-09 06:43:07.657200808 +0000 UTC m=+285.553618224 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s4mlq" (UID: "93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9") : configmap "kube-rbac-proxy" not found Sep 09 06:42:59.657440 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:42:59.657124 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s4mlq\" (UID: \"93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9\") " pod="openshift-insights/insights-runtime-extractor-s4mlq" Sep 09 06:42:59.657440 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:42:59.657251 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-insights-runtime-extractor-tls podName:93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9 nodeName:}" failed. No retries permitted until 2026-09-09 06:43:07.657241368 +0000 UTC m=+285.553658768 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s4mlq" (UID: "93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9") : secret "insights-runtime-extractor-tls" not found Sep 09 06:43:02.183872 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:43:02.183828 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-798d7b96c-6srkk" podUID="fe89f009-e9d3-404a-b39a-895d216f3c22" Sep 09 06:43:02.280453 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:02.280392 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/39fe02da-f881-42af-9eb7-35cc7e1a0c66-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cflhw\" (UID: \"39fe02da-f881-42af-9eb7-35cc7e1a0c66\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cflhw" Sep 09 06:43:02.280634 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:43:02.280553 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 09 06:43:02.280634 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:43:02.280631 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39fe02da-f881-42af-9eb7-35cc7e1a0c66-cluster-monitoring-operator-tls podName:39fe02da-f881-42af-9eb7-35cc7e1a0c66 nodeName:}" failed. No retries permitted until 2026-09-09 06:43:18.280612809 +0000 UTC m=+296.177030213 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/39fe02da-f881-42af-9eb7-35cc7e1a0c66-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-cflhw" (UID: "39fe02da-f881-42af-9eb7-35cc7e1a0c66") : secret "cluster-monitoring-operator-tls" not found Sep 09 06:43:02.506491 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:02.506417 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:43:05.910703 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:05.910663 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/635a2ebb-6998-49e2-8e4d-088725c2c079-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-5x9qz\" (UID: \"635a2ebb-6998-49e2-8e4d-088725c2c079\") " pod="openshift-network-console/networking-console-plugin-79977f9844-5x9qz" Sep 09 06:43:05.911118 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:05.910740 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-tls\") pod \"image-registry-798d7b96c-6srkk\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:43:05.913018 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:05.912997 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-tls\") pod \"image-registry-798d7b96c-6srkk\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:43:05.913160 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:05.913138 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/635a2ebb-6998-49e2-8e4d-088725c2c079-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-5x9qz\" (UID: \"635a2ebb-6998-49e2-8e4d-088725c2c079\") " pod="openshift-network-console/networking-console-plugin-79977f9844-5x9qz" Sep 09 06:43:06.011620 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:06.011586 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e0761f7-70d1-4952-b645-1b99f46e3fc7-metrics-tls\") pod \"dns-default-g54h7\" (UID: \"5e0761f7-70d1-4952-b645-1b99f46e3fc7\") " pod="openshift-dns/dns-default-g54h7" Sep 09 06:43:06.013732 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:06.013708 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e0761f7-70d1-4952-b645-1b99f46e3fc7-metrics-tls\") pod \"dns-default-g54h7\" (UID: \"5e0761f7-70d1-4952-b645-1b99f46e3fc7\") " pod="openshift-dns/dns-default-g54h7" Sep 09 06:43:06.101127 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:06.101098 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4pf4g\"" Sep 09 06:43:06.108993 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:06.108973 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-g54h7" Sep 09 06:43:06.109230 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:06.109216 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9h4tb\"" Sep 09 06:43:06.117124 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:06.117103 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:43:06.200495 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:06.200469 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-pb4cn\"" Sep 09 06:43:06.208576 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:06.208553 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-5x9qz" Sep 09 06:43:06.382992 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:06.382960 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-g54h7"] Sep 09 06:43:06.385617 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:43:06.385589 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e0761f7_70d1_4952_b645_1b99f46e3fc7.slice/crio-41dc5c95ebd2bb5b1a37c7f62b9a3a880d4defe6865d02c147410da218a928f3 WatchSource:0}: Error finding container 41dc5c95ebd2bb5b1a37c7f62b9a3a880d4defe6865d02c147410da218a928f3: Status 404 returned error can't find the container with id 41dc5c95ebd2bb5b1a37c7f62b9a3a880d4defe6865d02c147410da218a928f3 Sep 09 06:43:06.392944 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:06.392920 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-798d7b96c-6srkk"] Sep 09 06:43:06.395031 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:43:06.395008 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe89f009_e9d3_404a_b39a_895d216f3c22.slice/crio-060e3906253878f64cd53a98cc0ba54d3a8625e8dff79195da16b9793fb7cfde WatchSource:0}: Error finding container 060e3906253878f64cd53a98cc0ba54d3a8625e8dff79195da16b9793fb7cfde: Status 404 returned error can't find the container with id 060e3906253878f64cd53a98cc0ba54d3a8625e8dff79195da16b9793fb7cfde Sep 09 06:43:06.479486 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:06.479421 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-5x9qz"] Sep 09 06:43:06.482630 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:43:06.482607 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod635a2ebb_6998_49e2_8e4d_088725c2c079.slice/crio-847c91c70a6a4e840cce5dda015bc678045350bf1d6feaf8af3769c414013a28 WatchSource:0}: Error finding container 847c91c70a6a4e840cce5dda015bc678045350bf1d6feaf8af3769c414013a28: Status 404 returned error can't find the container with id 847c91c70a6a4e840cce5dda015bc678045350bf1d6feaf8af3769c414013a28 Sep 09 06:43:06.519520 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:06.519487 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g54h7" event={"ID":"5e0761f7-70d1-4952-b645-1b99f46e3fc7","Type":"ContainerStarted","Data":"41dc5c95ebd2bb5b1a37c7f62b9a3a880d4defe6865d02c147410da218a928f3"} Sep 09 06:43:06.520685 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:06.520656 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-5x9qz" event={"ID":"635a2ebb-6998-49e2-8e4d-088725c2c079","Type":"ContainerStarted","Data":"847c91c70a6a4e840cce5dda015bc678045350bf1d6feaf8af3769c414013a28"} Sep 09 06:43:06.522194 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:06.522171 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-798d7b96c-6srkk" event={"ID":"fe89f009-e9d3-404a-b39a-895d216f3c22","Type":"ContainerStarted","Data":"3e0e845c6775477961f2d1ef0f3488603ce73286bd8fc77dfa11ae217b3d51fa"} Sep 09 06:43:06.522283 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:06.522212 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-798d7b96c-6srkk" event={"ID":"fe89f009-e9d3-404a-b39a-895d216f3c22","Type":"ContainerStarted","Data":"060e3906253878f64cd53a98cc0ba54d3a8625e8dff79195da16b9793fb7cfde"} Sep 09 06:43:06.522283 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:06.522274 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:43:07.725606 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:07.725569 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s4mlq\" (UID: \"93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9\") " pod="openshift-insights/insights-runtime-extractor-s4mlq" Sep 09 06:43:07.725952 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:07.725644 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s4mlq\" (UID: \"93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9\") " pod="openshift-insights/insights-runtime-extractor-s4mlq" Sep 09 06:43:07.726193 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:07.726166 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s4mlq\" (UID: \"93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9\") " pod="openshift-insights/insights-runtime-extractor-s4mlq" Sep 09 06:43:07.727820 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:07.727798 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s4mlq\" (UID: \"93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9\") " pod="openshift-insights/insights-runtime-extractor-s4mlq" Sep 09 06:43:07.782952 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:07.782928 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-s4mlq" Sep 09 06:43:08.448137 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:08.447813 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-798d7b96c-6srkk" podStartSLOduration=285.447790336 podStartE2EDuration="4m45.447790336s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:38:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:43:06.546915558 +0000 UTC m=+284.443332981" watchObservedRunningTime="2026-09-09 06:43:08.447790336 +0000 UTC m=+286.344207759" Sep 09 06:43:08.448902 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:08.448875 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-s4mlq"] Sep 09 06:43:08.452365 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:43:08.452338 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93c58a55_9f3b_4b5d_a43b_4ff70ba6f3b9.slice/crio-ee8fc8d0820ee4569c8206126566a9b27bdbfcd73afd5ff1f7c3e86fbd146c89 WatchSource:0}: Error finding container ee8fc8d0820ee4569c8206126566a9b27bdbfcd73afd5ff1f7c3e86fbd146c89: Status 404 returned error can't find the container with id ee8fc8d0820ee4569c8206126566a9b27bdbfcd73afd5ff1f7c3e86fbd146c89 Sep 09 06:43:08.530436 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:08.530409 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g54h7" event={"ID":"5e0761f7-70d1-4952-b645-1b99f46e3fc7","Type":"ContainerStarted","Data":"a575dfc2eec77d0d643d48714066d0567007503d7d822afce8b5ab331e1e0571"} Sep 09 06:43:08.530540 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:08.530491 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g54h7" event={"ID":"5e0761f7-70d1-4952-b645-1b99f46e3fc7","Type":"ContainerStarted","Data":"46488eb6bebfdbdb9a40d0bfc2fcbf8add8f916eafc69f8578aa179766c290a2"} Sep 09 06:43:08.530601 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:08.530554 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-g54h7" Sep 09 06:43:08.532250 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:08.532223 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-5x9qz" event={"ID":"635a2ebb-6998-49e2-8e4d-088725c2c079","Type":"ContainerStarted","Data":"036ba000b441ce2b32c091de6f8959ec752bad3327a7f10d30e9fc2abf3e7363"} Sep 09 06:43:08.533727 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:08.533701 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s4mlq" event={"ID":"93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9","Type":"ContainerStarted","Data":"a8e85106c92af084074ef2de5266aa1840e54b96bbcd38ae5350820954abc046"} Sep 09 06:43:08.533822 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:08.533757 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s4mlq" event={"ID":"93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9","Type":"ContainerStarted","Data":"ee8fc8d0820ee4569c8206126566a9b27bdbfcd73afd5ff1f7c3e86fbd146c89"} Sep 09 06:43:08.551772 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:08.551730 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-g54h7" podStartSLOduration=251.705824589 podStartE2EDuration="4m13.551718658s" totalImagesPullingTime="1.845894069s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:38:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:43:08.551197203 +0000 UTC m=+286.447614631" watchObservedRunningTime="2026-09-09 06:43:08.551718658 +0000 UTC m=+286.448136079" Sep 09 06:43:09.538710 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:09.538676 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s4mlq" event={"ID":"93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9","Type":"ContainerStarted","Data":"da1297587b4a836d61866b83ac59c7684655b341cac9891f02c9be83814a0605"} Sep 09 06:43:10.543782 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:10.543750 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s4mlq" event={"ID":"93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9","Type":"ContainerStarted","Data":"91ba9a0d11bc1bfb9c4cb697d938cc70563f88f07ac20ee522212cc4ea366dc1"} Sep 09 06:43:10.571232 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:10.571186 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-5x9qz" podStartSLOduration=273.365069588 podStartE2EDuration="4m34.571171752s" totalImagesPullingTime="1.206102164s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:38:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:43:08.579222746 +0000 UTC m=+286.475640177" watchObservedRunningTime="2026-09-09 06:43:10.571171752 +0000 UTC m=+288.467589174" Sep 09 06:43:10.571370 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:10.571352 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-s4mlq" podStartSLOduration=17.737878269 podStartE2EDuration="19.571345544s" totalImagesPullingTime="1.833467275s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 06:42:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:43:10.570404699 +0000 UTC m=+288.466822121" watchObservedRunningTime="2026-09-09 06:43:10.571345544 +0000 UTC m=+288.467762966" Sep 09 06:43:10.696837 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:10.696812 2559 scope.go:117] "RemoveContainer" containerID="430b703e1122185a9834704455c8a7e5cc5d4a8b8c0f62e8ab37c85912cc43b9" Sep 09 06:43:11.548154 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:11.548128 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nf9gf_26d28788-4f98-454b-ab39-2d0d857aa674/console-operator/2.log" Sep 09 06:43:11.548576 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:11.548560 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nf9gf_26d28788-4f98-454b-ab39-2d0d857aa674/console-operator/1.log" Sep 09 06:43:11.548629 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:11.548592 2559 generic.go:356] "Generic (PLEG): container finished" podID="26d28788-4f98-454b-ab39-2d0d857aa674" containerID="d9ccc109dbca3b786db10a093e4e736f7cc9f6b821862085e361c1db818e0a8f" exitCode=255 Sep 09 06:43:11.548689 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:11.548671 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-nf9gf" event={"ID":"26d28788-4f98-454b-ab39-2d0d857aa674","Type":"ContainerDied","Data":"d9ccc109dbca3b786db10a093e4e736f7cc9f6b821862085e361c1db818e0a8f"} Sep 09 06:43:11.548735 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:11.548725 2559 scope.go:117] "RemoveContainer" containerID="430b703e1122185a9834704455c8a7e5cc5d4a8b8c0f62e8ab37c85912cc43b9" Sep 09 06:43:11.549117 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:11.549099 2559 scope.go:117] "RemoveContainer" containerID="d9ccc109dbca3b786db10a093e4e736f7cc9f6b821862085e361c1db818e0a8f" Sep 09 06:43:11.549285 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:43:11.549265 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-nf9gf_openshift-console-operator(26d28788-4f98-454b-ab39-2d0d857aa674)\"" pod="openshift-console-operator/console-operator-85b454874b-nf9gf" podUID="26d28788-4f98-454b-ab39-2d0d857aa674" Sep 09 06:43:12.553512 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:12.553480 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nf9gf_26d28788-4f98-454b-ab39-2d0d857aa674/console-operator/2.log" Sep 09 06:43:16.697663 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:43:16.697623 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-pkv72" podUID="336552c8-ca0f-434a-8353-bc61ea118fa5" Sep 09 06:43:16.854460 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:16.854427 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-nf9gf" Sep 09 06:43:16.854619 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:16.854490 2559 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-nf9gf" Sep 09 06:43:16.854874 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:16.854855 2559 scope.go:117] "RemoveContainer" containerID="d9ccc109dbca3b786db10a093e4e736f7cc9f6b821862085e361c1db818e0a8f" Sep 09 06:43:16.855082 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:43:16.855044 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-nf9gf_openshift-console-operator(26d28788-4f98-454b-ab39-2d0d857aa674)\"" pod="openshift-console-operator/console-operator-85b454874b-nf9gf" podUID="26d28788-4f98-454b-ab39-2d0d857aa674" Sep 09 06:43:18.295459 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:18.295424 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/39fe02da-f881-42af-9eb7-35cc7e1a0c66-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cflhw\" (UID: \"39fe02da-f881-42af-9eb7-35cc7e1a0c66\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cflhw" Sep 09 06:43:18.297793 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:18.297757 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/39fe02da-f881-42af-9eb7-35cc7e1a0c66-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cflhw\" (UID: \"39fe02da-f881-42af-9eb7-35cc7e1a0c66\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cflhw" Sep 09 06:43:18.540746 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:18.540717 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-g54h7" Sep 09 06:43:18.550154 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:18.550092 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cflhw" Sep 09 06:43:18.838779 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:18.838706 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cflhw"] Sep 09 06:43:18.845803 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:43:18.845775 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39fe02da_f881_42af_9eb7_35cc7e1a0c66.slice/crio-e78ad30433a786cff4dd0218aa378614757cac1eab5442db2ece093ec02480e7 WatchSource:0}: Error finding container e78ad30433a786cff4dd0218aa378614757cac1eab5442db2ece093ec02480e7: Status 404 returned error can't find the container with id e78ad30433a786cff4dd0218aa378614757cac1eab5442db2ece093ec02480e7 Sep 09 06:43:19.580406 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:19.580365 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cflhw" event={"ID":"39fe02da-f881-42af-9eb7-35cc7e1a0c66","Type":"ContainerStarted","Data":"e78ad30433a786cff4dd0218aa378614757cac1eab5442db2ece093ec02480e7"} Sep 09 06:43:20.586024 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:20.585995 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cflhw" event={"ID":"39fe02da-f881-42af-9eb7-35cc7e1a0c66","Type":"ContainerStarted","Data":"eaf533c96a51f3c002b6d079b365d7b3a93ae795093ce57f925c61c459628992"} Sep 09 06:43:20.627938 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:20.627896 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cflhw" podStartSLOduration=32.961684459 podStartE2EDuration="34.627881132s" totalImagesPullingTime="1.666196673s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:42:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:43:20.626771905 +0000 UTC m=+298.523189328" watchObservedRunningTime="2026-09-09 06:43:20.627881132 +0000 UTC m=+298.524298569" Sep 09 06:43:21.120854 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:21.120820 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-sx9jv"] Sep 09 06:43:21.123887 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:21.123872 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-sx9jv" Sep 09 06:43:21.127609 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:21.127586 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Sep 09 06:43:21.127838 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:21.127823 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-lg4pc\"" Sep 09 06:43:21.208238 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:21.208203 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-sx9jv"] Sep 09 06:43:21.217437 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:21.217417 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/43e1fde4-bef2-4106-afdd-585668465ba3-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-sx9jv\" (UID: \"43e1fde4-bef2-4106-afdd-585668465ba3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-sx9jv" Sep 09 06:43:21.318068 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:21.318022 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/43e1fde4-bef2-4106-afdd-585668465ba3-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-sx9jv\" (UID: \"43e1fde4-bef2-4106-afdd-585668465ba3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-sx9jv" Sep 09 06:43:21.320350 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:21.320326 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/43e1fde4-bef2-4106-afdd-585668465ba3-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-sx9jv\" (UID: \"43e1fde4-bef2-4106-afdd-585668465ba3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-sx9jv" Sep 09 06:43:21.432626 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:21.432590 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-sx9jv" Sep 09 06:43:21.703839 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:21.703761 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-sx9jv"] Sep 09 06:43:21.706680 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:43:21.706654 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43e1fde4_bef2_4106_afdd_585668465ba3.slice/crio-6138e0e9b4c2ad212ff1ab951b9e299c3780ec3afc2693a617470b994699c287 WatchSource:0}: Error finding container 6138e0e9b4c2ad212ff1ab951b9e299c3780ec3afc2693a617470b994699c287: Status 404 returned error can't find the container with id 6138e0e9b4c2ad212ff1ab951b9e299c3780ec3afc2693a617470b994699c287 Sep 09 06:43:22.467434 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:22.467409 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nf9gf_26d28788-4f98-454b-ab39-2d0d857aa674/console-operator/2.log" Sep 09 06:43:22.467434 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:22.467423 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nf9gf_26d28788-4f98-454b-ab39-2d0d857aa674/console-operator/2.log" Sep 09 06:43:22.483608 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:22.483582 2559 kubelet.go:1648] "Image garbage collection succeeded" Sep 09 06:43:22.594475 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:22.594337 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-sx9jv" event={"ID":"43e1fde4-bef2-4106-afdd-585668465ba3","Type":"ContainerStarted","Data":"6138e0e9b4c2ad212ff1ab951b9e299c3780ec3afc2693a617470b994699c287"} Sep 09 06:43:23.598871 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:23.598840 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-sx9jv" event={"ID":"43e1fde4-bef2-4106-afdd-585668465ba3","Type":"ContainerStarted","Data":"3929a525a60beef65627c641a29717d6c3200ccb4406ec5632f0f9dfb333784a"} Sep 09 06:43:23.599271 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:23.599026 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-sx9jv" Sep 09 06:43:23.603520 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:23.603496 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-sx9jv" Sep 09 06:43:23.615171 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:23.615130 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-sx9jv" podStartSLOduration=1.3647486930000001 podStartE2EDuration="2.615115924s" totalImagesPullingTime="1.250367231s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:43:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:43:23.613630689 +0000 UTC m=+301.510048110" watchObservedRunningTime="2026-09-09 06:43:23.615115924 +0000 UTC m=+301.511533346" Sep 09 06:43:24.165303 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:24.165267 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-wbww5"] Sep 09 06:43:24.191218 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:24.191191 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-wbww5"] Sep 09 06:43:24.191370 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:24.191288 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-wbww5" Sep 09 06:43:24.193752 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:24.193719 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 09 06:43:24.193888 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:24.193832 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 09 06:43:24.194710 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:24.194690 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-w2xl8\"" Sep 09 06:43:24.194812 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:24.194791 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 09 06:43:24.239069 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:24.239022 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8094df83-28f6-4432-94a7-80928df7d921-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-wbww5\" (UID: \"8094df83-28f6-4432-94a7-80928df7d921\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-wbww5" Sep 09 06:43:24.239238 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:24.239079 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8rkf\" (UniqueName: \"kubernetes.io/projected/8094df83-28f6-4432-94a7-80928df7d921-kube-api-access-d8rkf\") pod \"prometheus-operator-6f4946db56-wbww5\" (UID: \"8094df83-28f6-4432-94a7-80928df7d921\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-wbww5" Sep 09 06:43:24.239238 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:24.239099 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8094df83-28f6-4432-94a7-80928df7d921-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-wbww5\" (UID: \"8094df83-28f6-4432-94a7-80928df7d921\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-wbww5" Sep 09 06:43:24.239238 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:24.239184 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8094df83-28f6-4432-94a7-80928df7d921-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-wbww5\" (UID: \"8094df83-28f6-4432-94a7-80928df7d921\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-wbww5" Sep 09 06:43:24.340407 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:24.340368 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8094df83-28f6-4432-94a7-80928df7d921-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-wbww5\" (UID: \"8094df83-28f6-4432-94a7-80928df7d921\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-wbww5" Sep 09 06:43:24.340589 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:24.340422 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8rkf\" (UniqueName: \"kubernetes.io/projected/8094df83-28f6-4432-94a7-80928df7d921-kube-api-access-d8rkf\") pod \"prometheus-operator-6f4946db56-wbww5\" (UID: \"8094df83-28f6-4432-94a7-80928df7d921\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-wbww5" Sep 09 06:43:24.340589 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:24.340537 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8094df83-28f6-4432-94a7-80928df7d921-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-wbww5\" (UID: \"8094df83-28f6-4432-94a7-80928df7d921\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-wbww5" Sep 09 06:43:24.340711 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:24.340609 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8094df83-28f6-4432-94a7-80928df7d921-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-wbww5\" (UID: \"8094df83-28f6-4432-94a7-80928df7d921\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-wbww5" Sep 09 06:43:24.341140 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:24.341119 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8094df83-28f6-4432-94a7-80928df7d921-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-wbww5\" (UID: \"8094df83-28f6-4432-94a7-80928df7d921\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-wbww5" Sep 09 06:43:24.343085 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:24.343043 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8094df83-28f6-4432-94a7-80928df7d921-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-wbww5\" (UID: \"8094df83-28f6-4432-94a7-80928df7d921\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-wbww5" Sep 09 06:43:24.343183 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:24.343047 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8094df83-28f6-4432-94a7-80928df7d921-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-wbww5\" (UID: \"8094df83-28f6-4432-94a7-80928df7d921\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-wbww5" Sep 09 06:43:24.349444 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:24.349426 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8rkf\" (UniqueName: \"kubernetes.io/projected/8094df83-28f6-4432-94a7-80928df7d921-kube-api-access-d8rkf\") pod \"prometheus-operator-6f4946db56-wbww5\" (UID: \"8094df83-28f6-4432-94a7-80928df7d921\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-wbww5" Sep 09 06:43:24.500211 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:24.500186 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-wbww5" Sep 09 06:43:24.793780 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:24.793694 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-wbww5"] Sep 09 06:43:24.797193 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:43:24.797168 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8094df83_28f6_4432_94a7_80928df7d921.slice/crio-11aced6048eadbce78e8bc0e9bf089e026a47fd7ca72339f48bff1cd105e391b WatchSource:0}: Error finding container 11aced6048eadbce78e8bc0e9bf089e026a47fd7ca72339f48bff1cd105e391b: Status 404 returned error can't find the container with id 11aced6048eadbce78e8bc0e9bf089e026a47fd7ca72339f48bff1cd105e391b Sep 09 06:43:24.799234 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:24.799215 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 06:43:25.606850 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:25.606815 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-wbww5" event={"ID":"8094df83-28f6-4432-94a7-80928df7d921","Type":"ContainerStarted","Data":"11aced6048eadbce78e8bc0e9bf089e026a47fd7ca72339f48bff1cd105e391b"} Sep 09 06:43:26.121642 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:26.121593 2559 patch_prober.go:28] interesting pod/image-registry-798d7b96c-6srkk container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 06:43:26.122010 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:26.121660 2559 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-798d7b96c-6srkk" podUID="fe89f009-e9d3-404a-b39a-895d216f3c22" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 06:43:27.529429 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:27.529401 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:43:27.616736 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:27.616697 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-wbww5" event={"ID":"8094df83-28f6-4432-94a7-80928df7d921","Type":"ContainerStarted","Data":"01b0066d8a8739aea71a7cb1ae25ae8b7b77b63ebf187a9af41573270302ae78"} Sep 09 06:43:27.616902 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:27.616756 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-wbww5" event={"ID":"8094df83-28f6-4432-94a7-80928df7d921","Type":"ContainerStarted","Data":"a4c5976a4dbf746d7e86723f51be867a35e83460d832f7f7b5934fb2f741a469"} Sep 09 06:43:27.636608 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:27.636561 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-wbww5" podStartSLOduration=1.642769236 podStartE2EDuration="3.636543973s" totalImagesPullingTime="1.993774737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:43:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:43:27.635627507 +0000 UTC m=+305.532044932" watchObservedRunningTime="2026-09-09 06:43:27.636543973 +0000 UTC m=+305.532961395" Sep 09 06:43:29.565513 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:29.565480 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-xjl8t"] Sep 09 06:43:29.567677 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:29.567661 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xjl8t" Sep 09 06:43:29.570833 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:29.570803 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 09 06:43:29.570968 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:29.570895 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bvggj\"" Sep 09 06:43:29.570968 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:29.570947 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 09 06:43:29.571083 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:29.570947 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 09 06:43:29.684316 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:29.684281 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/11a1d01b-12f3-4e32-a60e-f0a9d8e2367a-node-exporter-textfile\") pod \"node-exporter-xjl8t\" (UID: \"11a1d01b-12f3-4e32-a60e-f0a9d8e2367a\") " pod="openshift-monitoring/node-exporter-xjl8t" Sep 09 06:43:29.684316 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:29.684315 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/11a1d01b-12f3-4e32-a60e-f0a9d8e2367a-metrics-client-ca\") pod \"node-exporter-xjl8t\" (UID: \"11a1d01b-12f3-4e32-a60e-f0a9d8e2367a\") " pod="openshift-monitoring/node-exporter-xjl8t" Sep 09 06:43:29.684529 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:29.684345 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/11a1d01b-12f3-4e32-a60e-f0a9d8e2367a-root\") pod \"node-exporter-xjl8t\" (UID: \"11a1d01b-12f3-4e32-a60e-f0a9d8e2367a\") " pod="openshift-monitoring/node-exporter-xjl8t" Sep 09 06:43:29.684529 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:29.684366 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/11a1d01b-12f3-4e32-a60e-f0a9d8e2367a-node-exporter-tls\") pod \"node-exporter-xjl8t\" (UID: \"11a1d01b-12f3-4e32-a60e-f0a9d8e2367a\") " pod="openshift-monitoring/node-exporter-xjl8t" Sep 09 06:43:29.684529 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:29.684388 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/11a1d01b-12f3-4e32-a60e-f0a9d8e2367a-node-exporter-accelerators-collector-config\") pod \"node-exporter-xjl8t\" (UID: \"11a1d01b-12f3-4e32-a60e-f0a9d8e2367a\") " pod="openshift-monitoring/node-exporter-xjl8t" Sep 09 06:43:29.684529 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:29.684420 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11a1d01b-12f3-4e32-a60e-f0a9d8e2367a-sys\") pod \"node-exporter-xjl8t\" (UID: \"11a1d01b-12f3-4e32-a60e-f0a9d8e2367a\") " pod="openshift-monitoring/node-exporter-xjl8t" Sep 09 06:43:29.684707 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:29.684529 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/11a1d01b-12f3-4e32-a60e-f0a9d8e2367a-node-exporter-wtmp\") pod \"node-exporter-xjl8t\" (UID: \"11a1d01b-12f3-4e32-a60e-f0a9d8e2367a\") " pod="openshift-monitoring/node-exporter-xjl8t" Sep 09 06:43:29.684707 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:29.684564 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/11a1d01b-12f3-4e32-a60e-f0a9d8e2367a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xjl8t\" (UID: \"11a1d01b-12f3-4e32-a60e-f0a9d8e2367a\") " pod="openshift-monitoring/node-exporter-xjl8t" Sep 09 06:43:29.684707 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:29.684591 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9975b\" (UniqueName: \"kubernetes.io/projected/11a1d01b-12f3-4e32-a60e-f0a9d8e2367a-kube-api-access-9975b\") pod \"node-exporter-xjl8t\" (UID: \"11a1d01b-12f3-4e32-a60e-f0a9d8e2367a\") " pod="openshift-monitoring/node-exporter-xjl8t" Sep 09 06:43:29.696531 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:29.696509 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv72" Sep 09 06:43:29.785183 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:29.785155 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/11a1d01b-12f3-4e32-a60e-f0a9d8e2367a-node-exporter-textfile\") pod \"node-exporter-xjl8t\" (UID: \"11a1d01b-12f3-4e32-a60e-f0a9d8e2367a\") " pod="openshift-monitoring/node-exporter-xjl8t" Sep 09 06:43:29.785183 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:29.785182 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/11a1d01b-12f3-4e32-a60e-f0a9d8e2367a-metrics-client-ca\") pod \"node-exporter-xjl8t\" (UID: \"11a1d01b-12f3-4e32-a60e-f0a9d8e2367a\") " pod="openshift-monitoring/node-exporter-xjl8t" Sep 09 06:43:29.785367 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:29.785207 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/11a1d01b-12f3-4e32-a60e-f0a9d8e2367a-root\") pod \"node-exporter-xjl8t\" (UID: \"11a1d01b-12f3-4e32-a60e-f0a9d8e2367a\") " pod="openshift-monitoring/node-exporter-xjl8t" Sep 09 06:43:29.785367 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:29.785225 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/11a1d01b-12f3-4e32-a60e-f0a9d8e2367a-node-exporter-tls\") pod \"node-exporter-xjl8t\" (UID: \"11a1d01b-12f3-4e32-a60e-f0a9d8e2367a\") " pod="openshift-monitoring/node-exporter-xjl8t" Sep 09 06:43:29.785367 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:29.785244 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/11a1d01b-12f3-4e32-a60e-f0a9d8e2367a-node-exporter-accelerators-collector-config\") pod \"node-exporter-xjl8t\" (UID: \"11a1d01b-12f3-4e32-a60e-f0a9d8e2367a\") " pod="openshift-monitoring/node-exporter-xjl8t" Sep 09 06:43:29.785367 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:29.785301 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/11a1d01b-12f3-4e32-a60e-f0a9d8e2367a-root\") pod \"node-exporter-xjl8t\" (UID: \"11a1d01b-12f3-4e32-a60e-f0a9d8e2367a\") " pod="openshift-monitoring/node-exporter-xjl8t" Sep 09 06:43:29.785367 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:29.785304 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11a1d01b-12f3-4e32-a60e-f0a9d8e2367a-sys\") pod \"node-exporter-xjl8t\" (UID: \"11a1d01b-12f3-4e32-a60e-f0a9d8e2367a\") " pod="openshift-monitoring/node-exporter-xjl8t" Sep 09 06:43:29.785367 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:29.785348 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11a1d01b-12f3-4e32-a60e-f0a9d8e2367a-sys\") pod \"node-exporter-xjl8t\" (UID: \"11a1d01b-12f3-4e32-a60e-f0a9d8e2367a\") " pod="openshift-monitoring/node-exporter-xjl8t" Sep 09 06:43:29.785659 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:29.785383 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/11a1d01b-12f3-4e32-a60e-f0a9d8e2367a-node-exporter-wtmp\") pod \"node-exporter-xjl8t\" (UID: \"11a1d01b-12f3-4e32-a60e-f0a9d8e2367a\") " pod="openshift-monitoring/node-exporter-xjl8t" Sep 09 06:43:29.785659 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:29.785411 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/11a1d01b-12f3-4e32-a60e-f0a9d8e2367a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xjl8t\" (UID: \"11a1d01b-12f3-4e32-a60e-f0a9d8e2367a\") " pod="openshift-monitoring/node-exporter-xjl8t" Sep 09 06:43:29.785659 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:29.785436 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9975b\" (UniqueName: \"kubernetes.io/projected/11a1d01b-12f3-4e32-a60e-f0a9d8e2367a-kube-api-access-9975b\") pod \"node-exporter-xjl8t\" (UID: \"11a1d01b-12f3-4e32-a60e-f0a9d8e2367a\") " pod="openshift-monitoring/node-exporter-xjl8t" Sep 09 06:43:29.785659 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:29.785542 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/11a1d01b-12f3-4e32-a60e-f0a9d8e2367a-node-exporter-wtmp\") pod \"node-exporter-xjl8t\" (UID: \"11a1d01b-12f3-4e32-a60e-f0a9d8e2367a\") " pod="openshift-monitoring/node-exporter-xjl8t" Sep 09 06:43:29.785659 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:29.785542 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/11a1d01b-12f3-4e32-a60e-f0a9d8e2367a-node-exporter-textfile\") pod \"node-exporter-xjl8t\" (UID: \"11a1d01b-12f3-4e32-a60e-f0a9d8e2367a\") " pod="openshift-monitoring/node-exporter-xjl8t" Sep 09 06:43:29.785926 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:29.785889 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/11a1d01b-12f3-4e32-a60e-f0a9d8e2367a-metrics-client-ca\") pod \"node-exporter-xjl8t\" (UID: \"11a1d01b-12f3-4e32-a60e-f0a9d8e2367a\") " pod="openshift-monitoring/node-exporter-xjl8t" Sep 09 06:43:29.785974 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:29.785923 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/11a1d01b-12f3-4e32-a60e-f0a9d8e2367a-node-exporter-accelerators-collector-config\") pod \"node-exporter-xjl8t\" (UID: \"11a1d01b-12f3-4e32-a60e-f0a9d8e2367a\") " pod="openshift-monitoring/node-exporter-xjl8t" Sep 09 06:43:29.787576 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:29.787550 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/11a1d01b-12f3-4e32-a60e-f0a9d8e2367a-node-exporter-tls\") pod \"node-exporter-xjl8t\" (UID: \"11a1d01b-12f3-4e32-a60e-f0a9d8e2367a\") " pod="openshift-monitoring/node-exporter-xjl8t" Sep 09 06:43:29.787668 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:29.787633 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/11a1d01b-12f3-4e32-a60e-f0a9d8e2367a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xjl8t\" (UID: \"11a1d01b-12f3-4e32-a60e-f0a9d8e2367a\") " pod="openshift-monitoring/node-exporter-xjl8t" Sep 09 06:43:29.793841 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:29.793818 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9975b\" (UniqueName: \"kubernetes.io/projected/11a1d01b-12f3-4e32-a60e-f0a9d8e2367a-kube-api-access-9975b\") pod \"node-exporter-xjl8t\" (UID: \"11a1d01b-12f3-4e32-a60e-f0a9d8e2367a\") " pod="openshift-monitoring/node-exporter-xjl8t" Sep 09 06:43:29.881446 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:29.881372 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xjl8t" Sep 09 06:43:29.892532 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:43:29.892506 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11a1d01b_12f3_4e32_a60e_f0a9d8e2367a.slice/crio-815c819317a51bdb87c4e83ed0ec2620f75357bb9f71c4a3b28b388d4116a72b WatchSource:0}: Error finding container 815c819317a51bdb87c4e83ed0ec2620f75357bb9f71c4a3b28b388d4116a72b: Status 404 returned error can't find the container with id 815c819317a51bdb87c4e83ed0ec2620f75357bb9f71c4a3b28b388d4116a72b Sep 09 06:43:30.621145 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.621107 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 06:43:30.624183 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.624159 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:30.628669 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.628623 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 09 06:43:30.628816 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.628704 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-8ldzq\"" Sep 09 06:43:30.628816 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.628777 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xjl8t" event={"ID":"11a1d01b-12f3-4e32-a60e-f0a9d8e2367a","Type":"ContainerStarted","Data":"815c819317a51bdb87c4e83ed0ec2620f75357bb9f71c4a3b28b388d4116a72b"} Sep 09 06:43:30.629009 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.628992 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 09 06:43:30.629737 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.629302 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 09 06:43:30.629737 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.629315 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 09 06:43:30.629737 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.629483 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 09 06:43:30.629737 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.629592 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 09 06:43:30.629737 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.629690 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 09 06:43:30.630132 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.629807 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 09 06:43:30.630132 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.630020 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 09 06:43:30.647119 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.647099 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 06:43:30.692841 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.692809 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/73fabe44-d13f-4841-8196-f0080f9d24e7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:30.692995 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.692856 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73fabe44-d13f-4841-8196-f0080f9d24e7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:30.692995 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.692892 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:30.692995 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.692921 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/73fabe44-d13f-4841-8196-f0080f9d24e7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:30.692995 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.692949 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-config-volume\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:30.692995 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.692984 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:30.693283 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.693014 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:30.693283 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.693125 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:30.693283 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.693159 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73fabe44-d13f-4841-8196-f0080f9d24e7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:30.693283 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.693196 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/73fabe44-d13f-4841-8196-f0080f9d24e7-config-out\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:30.693283 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.693264 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:30.693455 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.693333 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-web-config\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:30.693455 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.693373 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m47p9\" (UniqueName: \"kubernetes.io/projected/73fabe44-d13f-4841-8196-f0080f9d24e7-kube-api-access-m47p9\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:30.794695 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.794662 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:30.794874 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.794701 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/73fabe44-d13f-4841-8196-f0080f9d24e7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:30.794874 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.794830 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-config-volume\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:30.794989 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.794882 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:30.794989 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.794917 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:30.794989 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.794944 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:30.794989 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.794985 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73fabe44-d13f-4841-8196-f0080f9d24e7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:30.795227 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.795034 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/73fabe44-d13f-4841-8196-f0080f9d24e7-config-out\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:30.795227 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.795069 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:30.795227 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.795133 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-web-config\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:30.795227 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.795166 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m47p9\" (UniqueName: \"kubernetes.io/projected/73fabe44-d13f-4841-8196-f0080f9d24e7-kube-api-access-m47p9\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:30.795227 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.795206 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/73fabe44-d13f-4841-8196-f0080f9d24e7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:30.795456 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.795235 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73fabe44-d13f-4841-8196-f0080f9d24e7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:30.795612 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.795589 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/73fabe44-d13f-4841-8196-f0080f9d24e7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:30.797042 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:43:30.796480 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/73fabe44-d13f-4841-8196-f0080f9d24e7-alertmanager-trusted-ca-bundle podName:73fabe44-d13f-4841-8196-f0080f9d24e7 nodeName:}" failed. No retries permitted until 2026-09-09 06:43:31.296458148 +0000 UTC m=+309.192875564 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/73fabe44-d13f-4841-8196-f0080f9d24e7-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "73fabe44-d13f-4841-8196-f0080f9d24e7") : configmap references non-existent config key: ca-bundle.crt Sep 09 06:43:30.797782 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.797752 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73fabe44-d13f-4841-8196-f0080f9d24e7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:30.798421 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.798397 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:30.799205 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.799179 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:30.799486 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.799329 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-web-config\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:30.799954 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.799873 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:30.799954 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.799887 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:30.800150 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.800086 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-config-volume\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:30.800208 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.800147 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/73fabe44-d13f-4841-8196-f0080f9d24e7-config-out\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:30.800253 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.800235 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/73fabe44-d13f-4841-8196-f0080f9d24e7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:30.801411 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.801383 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:30.811457 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:30.811437 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m47p9\" (UniqueName: \"kubernetes.io/projected/73fabe44-d13f-4841-8196-f0080f9d24e7-kube-api-access-m47p9\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:31.299353 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:31.299279 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73fabe44-d13f-4841-8196-f0080f9d24e7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:31.300101 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:31.300081 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73fabe44-d13f-4841-8196-f0080f9d24e7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:31.536739 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:31.536683 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:43:31.634384 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:31.634349 2559 generic.go:356] "Generic (PLEG): container finished" podID="11a1d01b-12f3-4e32-a60e-f0a9d8e2367a" containerID="7229a3eeebb4961ff5f00f3e74225173994e01bd36d5d5a862fdb5e134d1d324" exitCode=0 Sep 09 06:43:31.634735 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:31.634429 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xjl8t" event={"ID":"11a1d01b-12f3-4e32-a60e-f0a9d8e2367a","Type":"ContainerDied","Data":"7229a3eeebb4961ff5f00f3e74225173994e01bd36d5d5a862fdb5e134d1d324"} Sep 09 06:43:31.701366 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:31.697475 2559 scope.go:117] "RemoveContainer" containerID="d9ccc109dbca3b786db10a093e4e736f7cc9f6b821862085e361c1db818e0a8f" Sep 09 06:43:31.847165 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:31.847092 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 06:43:31.850246 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:43:31.850204 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73fabe44_d13f_4841_8196_f0080f9d24e7.slice/crio-3681098c91c6489758e6fee3ad31c01abd6f9cd99e4bdc82bc4fa2592f29ca1a WatchSource:0}: Error finding container 3681098c91c6489758e6fee3ad31c01abd6f9cd99e4bdc82bc4fa2592f29ca1a: Status 404 returned error can't find the container with id 3681098c91c6489758e6fee3ad31c01abd6f9cd99e4bdc82bc4fa2592f29ca1a Sep 09 06:43:32.640271 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:32.640233 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"73fabe44-d13f-4841-8196-f0080f9d24e7","Type":"ContainerStarted","Data":"3681098c91c6489758e6fee3ad31c01abd6f9cd99e4bdc82bc4fa2592f29ca1a"} Sep 09 06:43:32.642656 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:32.642634 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nf9gf_26d28788-4f98-454b-ab39-2d0d857aa674/console-operator/2.log" Sep 09 06:43:32.642780 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:32.642746 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-nf9gf" event={"ID":"26d28788-4f98-454b-ab39-2d0d857aa674","Type":"ContainerStarted","Data":"10c84616b58abaa84670618698bb2c7c1c3a36f483d01b21a3e11e84fb8575b0"} Sep 09 06:43:32.643119 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:32.643044 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-nf9gf" Sep 09 06:43:32.645799 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:32.645773 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xjl8t" event={"ID":"11a1d01b-12f3-4e32-a60e-f0a9d8e2367a","Type":"ContainerStarted","Data":"6e0c549b3dc8d66b42dfcaed19f3d954f1edbf976873b5b5a413075cda3a73d1"} Sep 09 06:43:32.645906 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:32.645816 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xjl8t" event={"ID":"11a1d01b-12f3-4e32-a60e-f0a9d8e2367a","Type":"ContainerStarted","Data":"e1c11556d7616544641cfb3b4e465d12615661049fb1b0cac5d10006868245f1"} Sep 09 06:43:32.688446 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:32.688386 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-nf9gf" podStartSLOduration=43.660000601 podStartE2EDuration="46.688366196s" totalImagesPullingTime="3.028365595s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:42:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:43:32.686006293 +0000 UTC m=+310.582423716" watchObservedRunningTime="2026-09-09 06:43:32.688366196 +0000 UTC m=+310.584783619" Sep 09 06:43:32.721638 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:32.721579 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-xjl8t" podStartSLOduration=2.534939146 podStartE2EDuration="3.721558213s" totalImagesPullingTime="1.125643296s" totalInitContainerRuntime="60.975771ms" isStatefulPod=true podCreationTimestamp="2026-09-09 06:43:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:43:32.719543319 +0000 UTC m=+310.615960743" watchObservedRunningTime="2026-09-09 06:43:32.721558213 +0000 UTC m=+310.617975701" Sep 09 06:43:33.004104 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.004083 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-nf9gf" Sep 09 06:43:33.208909 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.208871 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-4zgn6"] Sep 09 06:43:33.210934 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.210911 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-4zgn6" Sep 09 06:43:33.213645 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.213622 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 09 06:43:33.213736 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.213672 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 09 06:43:33.213785 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.213748 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-jfrb2\"" Sep 09 06:43:33.229844 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.229821 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-4zgn6"] Sep 09 06:43:33.317652 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.317584 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mq7bs\" (UniqueName: \"kubernetes.io/projected/f1089629-f49d-4225-8f67-332ea17548ca-kube-api-access-mq7bs\") pod \"downloads-85d98c998f-4zgn6\" (UID: \"f1089629-f49d-4225-8f67-332ea17548ca\") " pod="openshift-console/downloads-85d98c998f-4zgn6" Sep 09 06:43:33.418406 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.418372 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mq7bs\" (UniqueName: \"kubernetes.io/projected/f1089629-f49d-4225-8f67-332ea17548ca-kube-api-access-mq7bs\") pod \"downloads-85d98c998f-4zgn6\" (UID: \"f1089629-f49d-4225-8f67-332ea17548ca\") " pod="openshift-console/downloads-85d98c998f-4zgn6" Sep 09 06:43:33.427973 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.427948 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mq7bs\" (UniqueName: \"kubernetes.io/projected/f1089629-f49d-4225-8f67-332ea17548ca-kube-api-access-mq7bs\") pod \"downloads-85d98c998f-4zgn6\" (UID: \"f1089629-f49d-4225-8f67-332ea17548ca\") " pod="openshift-console/downloads-85d98c998f-4zgn6" Sep 09 06:43:33.519526 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.519500 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-4zgn6" Sep 09 06:43:33.651330 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.651300 2559 generic.go:356] "Generic (PLEG): container finished" podID="73fabe44-d13f-4841-8196-f0080f9d24e7" containerID="3a430b79605ba0a8116bc672e81d1946fbb69ad45700702aa2b7eb8ed738a5c2" exitCode=0 Sep 09 06:43:33.651805 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.651393 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"73fabe44-d13f-4841-8196-f0080f9d24e7","Type":"ContainerDied","Data":"3a430b79605ba0a8116bc672e81d1946fbb69ad45700702aa2b7eb8ed738a5c2"} Sep 09 06:43:33.790258 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.790224 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-78b989bfc-cl4hl"] Sep 09 06:43:33.793322 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.793305 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-78b989bfc-cl4hl" Sep 09 06:43:33.799366 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.799342 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Sep 09 06:43:33.799789 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.799767 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Sep 09 06:43:33.799989 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.799974 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Sep 09 06:43:33.800040 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.800017 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Sep 09 06:43:33.800697 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.800674 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Sep 09 06:43:33.800792 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.800758 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-d9gwv\"" Sep 09 06:43:33.802127 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.802111 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-bqipljgo360na\"" Sep 09 06:43:33.808687 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.808666 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-4zgn6"] Sep 09 06:43:33.811255 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:43:33.811233 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1089629_f49d_4225_8f67_332ea17548ca.slice/crio-3dab9614e4f863d5dfcfa01eb40e77c5ad04dccca06e1b2faa39936798754d1a WatchSource:0}: Error finding container 3dab9614e4f863d5dfcfa01eb40e77c5ad04dccca06e1b2faa39936798754d1a: Status 404 returned error can't find the container with id 3dab9614e4f863d5dfcfa01eb40e77c5ad04dccca06e1b2faa39936798754d1a Sep 09 06:43:33.818286 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.818265 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-78b989bfc-cl4hl"] Sep 09 06:43:33.822084 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.822028 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/694195a6-fedc-4d83-9cfd-bef2072219b9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-78b989bfc-cl4hl\" (UID: \"694195a6-fedc-4d83-9cfd-bef2072219b9\") " pod="openshift-monitoring/thanos-querier-78b989bfc-cl4hl" Sep 09 06:43:33.822084 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.822077 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/694195a6-fedc-4d83-9cfd-bef2072219b9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-78b989bfc-cl4hl\" (UID: \"694195a6-fedc-4d83-9cfd-bef2072219b9\") " pod="openshift-monitoring/thanos-querier-78b989bfc-cl4hl" Sep 09 06:43:33.822237 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.822176 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/694195a6-fedc-4d83-9cfd-bef2072219b9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-78b989bfc-cl4hl\" (UID: \"694195a6-fedc-4d83-9cfd-bef2072219b9\") " pod="openshift-monitoring/thanos-querier-78b989bfc-cl4hl" Sep 09 06:43:33.822237 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.822211 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/694195a6-fedc-4d83-9cfd-bef2072219b9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-78b989bfc-cl4hl\" (UID: \"694195a6-fedc-4d83-9cfd-bef2072219b9\") " pod="openshift-monitoring/thanos-querier-78b989bfc-cl4hl" Sep 09 06:43:33.822341 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.822241 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/694195a6-fedc-4d83-9cfd-bef2072219b9-secret-thanos-querier-tls\") pod \"thanos-querier-78b989bfc-cl4hl\" (UID: \"694195a6-fedc-4d83-9cfd-bef2072219b9\") " pod="openshift-monitoring/thanos-querier-78b989bfc-cl4hl" Sep 09 06:43:33.822341 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.822279 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tv6q2\" (UniqueName: \"kubernetes.io/projected/694195a6-fedc-4d83-9cfd-bef2072219b9-kube-api-access-tv6q2\") pod \"thanos-querier-78b989bfc-cl4hl\" (UID: \"694195a6-fedc-4d83-9cfd-bef2072219b9\") " pod="openshift-monitoring/thanos-querier-78b989bfc-cl4hl" Sep 09 06:43:33.822341 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.822320 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/694195a6-fedc-4d83-9cfd-bef2072219b9-metrics-client-ca\") pod \"thanos-querier-78b989bfc-cl4hl\" (UID: \"694195a6-fedc-4d83-9cfd-bef2072219b9\") " pod="openshift-monitoring/thanos-querier-78b989bfc-cl4hl" Sep 09 06:43:33.822466 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.822348 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/694195a6-fedc-4d83-9cfd-bef2072219b9-secret-grpc-tls\") pod \"thanos-querier-78b989bfc-cl4hl\" (UID: \"694195a6-fedc-4d83-9cfd-bef2072219b9\") " pod="openshift-monitoring/thanos-querier-78b989bfc-cl4hl" Sep 09 06:43:33.923673 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.923641 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/694195a6-fedc-4d83-9cfd-bef2072219b9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-78b989bfc-cl4hl\" (UID: \"694195a6-fedc-4d83-9cfd-bef2072219b9\") " pod="openshift-monitoring/thanos-querier-78b989bfc-cl4hl" Sep 09 06:43:33.923808 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.923682 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/694195a6-fedc-4d83-9cfd-bef2072219b9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-78b989bfc-cl4hl\" (UID: \"694195a6-fedc-4d83-9cfd-bef2072219b9\") " pod="openshift-monitoring/thanos-querier-78b989bfc-cl4hl" Sep 09 06:43:33.923808 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.923723 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/694195a6-fedc-4d83-9cfd-bef2072219b9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-78b989bfc-cl4hl\" (UID: \"694195a6-fedc-4d83-9cfd-bef2072219b9\") " pod="openshift-monitoring/thanos-querier-78b989bfc-cl4hl" Sep 09 06:43:33.923808 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.923738 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/694195a6-fedc-4d83-9cfd-bef2072219b9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-78b989bfc-cl4hl\" (UID: \"694195a6-fedc-4d83-9cfd-bef2072219b9\") " pod="openshift-monitoring/thanos-querier-78b989bfc-cl4hl" Sep 09 06:43:33.923808 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.923757 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/694195a6-fedc-4d83-9cfd-bef2072219b9-secret-thanos-querier-tls\") pod \"thanos-querier-78b989bfc-cl4hl\" (UID: \"694195a6-fedc-4d83-9cfd-bef2072219b9\") " pod="openshift-monitoring/thanos-querier-78b989bfc-cl4hl" Sep 09 06:43:33.923808 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.923778 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv6q2\" (UniqueName: \"kubernetes.io/projected/694195a6-fedc-4d83-9cfd-bef2072219b9-kube-api-access-tv6q2\") pod \"thanos-querier-78b989bfc-cl4hl\" (UID: \"694195a6-fedc-4d83-9cfd-bef2072219b9\") " pod="openshift-monitoring/thanos-querier-78b989bfc-cl4hl" Sep 09 06:43:33.923808 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.923800 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/694195a6-fedc-4d83-9cfd-bef2072219b9-metrics-client-ca\") pod \"thanos-querier-78b989bfc-cl4hl\" (UID: \"694195a6-fedc-4d83-9cfd-bef2072219b9\") " pod="openshift-monitoring/thanos-querier-78b989bfc-cl4hl" Sep 09 06:43:33.924145 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.923818 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/694195a6-fedc-4d83-9cfd-bef2072219b9-secret-grpc-tls\") pod \"thanos-querier-78b989bfc-cl4hl\" (UID: \"694195a6-fedc-4d83-9cfd-bef2072219b9\") " pod="openshift-monitoring/thanos-querier-78b989bfc-cl4hl" Sep 09 06:43:33.924714 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.924664 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/694195a6-fedc-4d83-9cfd-bef2072219b9-metrics-client-ca\") pod \"thanos-querier-78b989bfc-cl4hl\" (UID: \"694195a6-fedc-4d83-9cfd-bef2072219b9\") " pod="openshift-monitoring/thanos-querier-78b989bfc-cl4hl" Sep 09 06:43:33.926295 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.926255 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/694195a6-fedc-4d83-9cfd-bef2072219b9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-78b989bfc-cl4hl\" (UID: \"694195a6-fedc-4d83-9cfd-bef2072219b9\") " pod="openshift-monitoring/thanos-querier-78b989bfc-cl4hl" Sep 09 06:43:33.926391 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.926355 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/694195a6-fedc-4d83-9cfd-bef2072219b9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-78b989bfc-cl4hl\" (UID: \"694195a6-fedc-4d83-9cfd-bef2072219b9\") " pod="openshift-monitoring/thanos-querier-78b989bfc-cl4hl" Sep 09 06:43:33.926719 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.926696 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/694195a6-fedc-4d83-9cfd-bef2072219b9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-78b989bfc-cl4hl\" (UID: \"694195a6-fedc-4d83-9cfd-bef2072219b9\") " pod="openshift-monitoring/thanos-querier-78b989bfc-cl4hl" Sep 09 06:43:33.926801 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.926700 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/694195a6-fedc-4d83-9cfd-bef2072219b9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-78b989bfc-cl4hl\" (UID: \"694195a6-fedc-4d83-9cfd-bef2072219b9\") " pod="openshift-monitoring/thanos-querier-78b989bfc-cl4hl" Sep 09 06:43:33.926801 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.926758 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/694195a6-fedc-4d83-9cfd-bef2072219b9-secret-thanos-querier-tls\") pod \"thanos-querier-78b989bfc-cl4hl\" (UID: \"694195a6-fedc-4d83-9cfd-bef2072219b9\") " pod="openshift-monitoring/thanos-querier-78b989bfc-cl4hl" Sep 09 06:43:33.926801 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.926790 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/694195a6-fedc-4d83-9cfd-bef2072219b9-secret-grpc-tls\") pod \"thanos-querier-78b989bfc-cl4hl\" (UID: \"694195a6-fedc-4d83-9cfd-bef2072219b9\") " pod="openshift-monitoring/thanos-querier-78b989bfc-cl4hl" Sep 09 06:43:33.941665 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:33.941640 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv6q2\" (UniqueName: \"kubernetes.io/projected/694195a6-fedc-4d83-9cfd-bef2072219b9-kube-api-access-tv6q2\") pod \"thanos-querier-78b989bfc-cl4hl\" (UID: \"694195a6-fedc-4d83-9cfd-bef2072219b9\") " pod="openshift-monitoring/thanos-querier-78b989bfc-cl4hl" Sep 09 06:43:34.103748 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:34.103677 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-78b989bfc-cl4hl" Sep 09 06:43:34.415237 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:34.415152 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-78b989bfc-cl4hl"] Sep 09 06:43:34.418924 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:43:34.418875 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod694195a6_fedc_4d83_9cfd_bef2072219b9.slice/crio-49fb558a8b25545b08e964f86ab71ea842b8b5f9e5e29d261c9ebe7b2e1ce6f5 WatchSource:0}: Error finding container 49fb558a8b25545b08e964f86ab71ea842b8b5f9e5e29d261c9ebe7b2e1ce6f5: Status 404 returned error can't find the container with id 49fb558a8b25545b08e964f86ab71ea842b8b5f9e5e29d261c9ebe7b2e1ce6f5 Sep 09 06:43:34.658839 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:34.658797 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78b989bfc-cl4hl" event={"ID":"694195a6-fedc-4d83-9cfd-bef2072219b9","Type":"ContainerStarted","Data":"49fb558a8b25545b08e964f86ab71ea842b8b5f9e5e29d261c9ebe7b2e1ce6f5"} Sep 09 06:43:34.661039 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:34.661007 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-4zgn6" event={"ID":"f1089629-f49d-4225-8f67-332ea17548ca","Type":"ContainerStarted","Data":"3dab9614e4f863d5dfcfa01eb40e77c5ad04dccca06e1b2faa39936798754d1a"} Sep 09 06:43:35.669516 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:35.669482 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"73fabe44-d13f-4841-8196-f0080f9d24e7","Type":"ContainerStarted","Data":"c7bc791eae25fd65c248934ca459991af30fb3de2d9fa6f03c10fb4c265aad9e"} Sep 09 06:43:36.680101 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:36.680041 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"73fabe44-d13f-4841-8196-f0080f9d24e7","Type":"ContainerStarted","Data":"e4d75f04171b0e9f4697e5e1e48aed8f62780b4a3a4a15a970ee7aa9d7c1d6d8"} Sep 09 06:43:36.680576 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:36.680194 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"73fabe44-d13f-4841-8196-f0080f9d24e7","Type":"ContainerStarted","Data":"8841b40bf8c2f336cf932c0d3b8d8a9bfdb7c0a36289552a218e71d8ed56a900"} Sep 09 06:43:36.680576 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:36.680252 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"73fabe44-d13f-4841-8196-f0080f9d24e7","Type":"ContainerStarted","Data":"8d57f23a0f1a81fbe6cc3e89d2040e38766e467e321966da0ff99c05c084ea12"} Sep 09 06:43:36.680576 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:36.680294 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"73fabe44-d13f-4841-8196-f0080f9d24e7","Type":"ContainerStarted","Data":"ca80b536126fb47e88260039ec65b485c620bf172b32afc31718e8d38e0988f1"} Sep 09 06:43:37.688097 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:37.687940 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78b989bfc-cl4hl" event={"ID":"694195a6-fedc-4d83-9cfd-bef2072219b9","Type":"ContainerStarted","Data":"b8b8ca837b8d05b71382b0cd20ccbbe2a546088413d54477ca5a953b97b4d7ef"} Sep 09 06:43:37.688097 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:37.688029 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78b989bfc-cl4hl" event={"ID":"694195a6-fedc-4d83-9cfd-bef2072219b9","Type":"ContainerStarted","Data":"a3c4f25393bf4dc44c4159893525f53c0d864f8c4a0d0d325b926c768dcb6b13"} Sep 09 06:43:37.688097 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:37.688089 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78b989bfc-cl4hl" event={"ID":"694195a6-fedc-4d83-9cfd-bef2072219b9","Type":"ContainerStarted","Data":"6317cd8ca5cff71590c88d2ffbf88d62623f7c26c85cffce5fdf7e7fe7d320d7"} Sep 09 06:43:37.692319 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:37.692282 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"73fabe44-d13f-4841-8196-f0080f9d24e7","Type":"ContainerStarted","Data":"8096172dd8bb7806ba753d2550c562a904e31e2bc48661fe4ffea37c532bb645"} Sep 09 06:43:37.708181 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:37.708154 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-798d7b96c-6srkk"] Sep 09 06:43:37.759631 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:37.759587 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.246729789 podStartE2EDuration="7.759567177s" totalImagesPullingTime="4.494905767s" totalInitContainerRuntime="17.931621ms" isStatefulPod=false podCreationTimestamp="2026-09-09 06:43:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:43:37.758207942 +0000 UTC m=+315.654625365" watchObservedRunningTime="2026-09-09 06:43:37.759567177 +0000 UTC m=+315.655984598" Sep 09 06:43:38.706660 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:38.706613 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78b989bfc-cl4hl" event={"ID":"694195a6-fedc-4d83-9cfd-bef2072219b9","Type":"ContainerStarted","Data":"c146c4d84487380b56962eaa534cef907b5bf327fada21bb219205e355d62e4c"} Sep 09 06:43:38.707141 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:38.706704 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78b989bfc-cl4hl" event={"ID":"694195a6-fedc-4d83-9cfd-bef2072219b9","Type":"ContainerStarted","Data":"4b5d17e47a806ca6485699b826612b696db2d2da43c70a5598862ed67b32d941"} Sep 09 06:43:38.707141 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:38.706747 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78b989bfc-cl4hl" event={"ID":"694195a6-fedc-4d83-9cfd-bef2072219b9","Type":"ContainerStarted","Data":"fbfd6105aed4a06147a614338936bb79b624dcdd86064a0f719ef5c477427d69"} Sep 09 06:43:38.707141 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:38.707100 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-78b989bfc-cl4hl" Sep 09 06:43:38.748458 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:38.748158 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-78b989bfc-cl4hl" podStartSLOduration=2.777316605 podStartE2EDuration="5.748141112s" totalImagesPullingTime="2.970824507s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:43:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:43:38.746821958 +0000 UTC m=+316.643239381" watchObservedRunningTime="2026-09-09 06:43:38.748141112 +0000 UTC m=+316.644558530" Sep 09 06:43:42.599116 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:42.599081 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-54b48b5d89-5qhd2"] Sep 09 06:43:42.603315 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:42.603286 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54b48b5d89-5qhd2" Sep 09 06:43:42.606495 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:42.606445 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 09 06:43:42.606611 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:42.606493 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 09 06:43:42.607363 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:42.607338 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 09 06:43:42.607463 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:42.607399 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 09 06:43:42.607525 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:42.607338 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-58spp\"" Sep 09 06:43:42.607585 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:42.607340 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 09 06:43:42.615613 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:42.615594 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54b48b5d89-5qhd2"] Sep 09 06:43:42.714410 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:42.714376 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7580ab0d-f488-4191-9eea-b8d56bc21ded-console-oauth-config\") pod \"console-54b48b5d89-5qhd2\" (UID: \"7580ab0d-f488-4191-9eea-b8d56bc21ded\") " pod="openshift-console/console-54b48b5d89-5qhd2" Sep 09 06:43:42.714410 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:42.714413 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmghn\" (UniqueName: \"kubernetes.io/projected/7580ab0d-f488-4191-9eea-b8d56bc21ded-kube-api-access-hmghn\") pod \"console-54b48b5d89-5qhd2\" (UID: \"7580ab0d-f488-4191-9eea-b8d56bc21ded\") " pod="openshift-console/console-54b48b5d89-5qhd2" Sep 09 06:43:42.714656 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:42.714528 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7580ab0d-f488-4191-9eea-b8d56bc21ded-console-serving-cert\") pod \"console-54b48b5d89-5qhd2\" (UID: \"7580ab0d-f488-4191-9eea-b8d56bc21ded\") " pod="openshift-console/console-54b48b5d89-5qhd2" Sep 09 06:43:42.714656 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:42.714582 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7580ab0d-f488-4191-9eea-b8d56bc21ded-service-ca\") pod \"console-54b48b5d89-5qhd2\" (UID: \"7580ab0d-f488-4191-9eea-b8d56bc21ded\") " pod="openshift-console/console-54b48b5d89-5qhd2" Sep 09 06:43:42.714762 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:42.714688 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7580ab0d-f488-4191-9eea-b8d56bc21ded-console-config\") pod \"console-54b48b5d89-5qhd2\" (UID: \"7580ab0d-f488-4191-9eea-b8d56bc21ded\") " pod="openshift-console/console-54b48b5d89-5qhd2" Sep 09 06:43:42.714762 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:42.714725 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7580ab0d-f488-4191-9eea-b8d56bc21ded-oauth-serving-cert\") pod \"console-54b48b5d89-5qhd2\" (UID: \"7580ab0d-f488-4191-9eea-b8d56bc21ded\") " pod="openshift-console/console-54b48b5d89-5qhd2" Sep 09 06:43:42.815450 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:42.815406 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7580ab0d-f488-4191-9eea-b8d56bc21ded-console-oauth-config\") pod \"console-54b48b5d89-5qhd2\" (UID: \"7580ab0d-f488-4191-9eea-b8d56bc21ded\") " pod="openshift-console/console-54b48b5d89-5qhd2" Sep 09 06:43:42.815450 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:42.815451 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmghn\" (UniqueName: \"kubernetes.io/projected/7580ab0d-f488-4191-9eea-b8d56bc21ded-kube-api-access-hmghn\") pod \"console-54b48b5d89-5qhd2\" (UID: \"7580ab0d-f488-4191-9eea-b8d56bc21ded\") " pod="openshift-console/console-54b48b5d89-5qhd2" Sep 09 06:43:42.815690 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:42.815503 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7580ab0d-f488-4191-9eea-b8d56bc21ded-console-serving-cert\") pod \"console-54b48b5d89-5qhd2\" (UID: \"7580ab0d-f488-4191-9eea-b8d56bc21ded\") " pod="openshift-console/console-54b48b5d89-5qhd2" Sep 09 06:43:42.815690 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:42.815527 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7580ab0d-f488-4191-9eea-b8d56bc21ded-service-ca\") pod \"console-54b48b5d89-5qhd2\" (UID: \"7580ab0d-f488-4191-9eea-b8d56bc21ded\") " pod="openshift-console/console-54b48b5d89-5qhd2" Sep 09 06:43:42.815690 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:42.815574 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7580ab0d-f488-4191-9eea-b8d56bc21ded-console-config\") pod \"console-54b48b5d89-5qhd2\" (UID: \"7580ab0d-f488-4191-9eea-b8d56bc21ded\") " pod="openshift-console/console-54b48b5d89-5qhd2" Sep 09 06:43:42.815690 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:42.815599 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7580ab0d-f488-4191-9eea-b8d56bc21ded-oauth-serving-cert\") pod \"console-54b48b5d89-5qhd2\" (UID: \"7580ab0d-f488-4191-9eea-b8d56bc21ded\") " pod="openshift-console/console-54b48b5d89-5qhd2" Sep 09 06:43:42.816514 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:42.816472 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7580ab0d-f488-4191-9eea-b8d56bc21ded-console-config\") pod \"console-54b48b5d89-5qhd2\" (UID: \"7580ab0d-f488-4191-9eea-b8d56bc21ded\") " pod="openshift-console/console-54b48b5d89-5qhd2" Sep 09 06:43:42.816514 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:42.816509 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7580ab0d-f488-4191-9eea-b8d56bc21ded-service-ca\") pod \"console-54b48b5d89-5qhd2\" (UID: \"7580ab0d-f488-4191-9eea-b8d56bc21ded\") " pod="openshift-console/console-54b48b5d89-5qhd2" Sep 09 06:43:42.816748 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:42.816509 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7580ab0d-f488-4191-9eea-b8d56bc21ded-oauth-serving-cert\") pod \"console-54b48b5d89-5qhd2\" (UID: \"7580ab0d-f488-4191-9eea-b8d56bc21ded\") " pod="openshift-console/console-54b48b5d89-5qhd2" Sep 09 06:43:42.818260 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:42.818235 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7580ab0d-f488-4191-9eea-b8d56bc21ded-console-oauth-config\") pod \"console-54b48b5d89-5qhd2\" (UID: \"7580ab0d-f488-4191-9eea-b8d56bc21ded\") " pod="openshift-console/console-54b48b5d89-5qhd2" Sep 09 06:43:42.818658 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:42.818641 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7580ab0d-f488-4191-9eea-b8d56bc21ded-console-serving-cert\") pod \"console-54b48b5d89-5qhd2\" (UID: \"7580ab0d-f488-4191-9eea-b8d56bc21ded\") " pod="openshift-console/console-54b48b5d89-5qhd2" Sep 09 06:43:42.824306 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:42.824282 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmghn\" (UniqueName: \"kubernetes.io/projected/7580ab0d-f488-4191-9eea-b8d56bc21ded-kube-api-access-hmghn\") pod \"console-54b48b5d89-5qhd2\" (UID: \"7580ab0d-f488-4191-9eea-b8d56bc21ded\") " pod="openshift-console/console-54b48b5d89-5qhd2" Sep 09 06:43:42.914427 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:42.914335 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54b48b5d89-5qhd2" Sep 09 06:43:43.195686 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:43.195661 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54b48b5d89-5qhd2"] Sep 09 06:43:43.199479 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:43:43.199445 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7580ab0d_f488_4191_9eea_b8d56bc21ded.slice/crio-85c9bec4e43bcd9ada796efec77f84abb91756fc6a076fbde8d0dac729b99cc4 WatchSource:0}: Error finding container 85c9bec4e43bcd9ada796efec77f84abb91756fc6a076fbde8d0dac729b99cc4: Status 404 returned error can't find the container with id 85c9bec4e43bcd9ada796efec77f84abb91756fc6a076fbde8d0dac729b99cc4 Sep 09 06:43:43.730151 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:43.730113 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54b48b5d89-5qhd2" event={"ID":"7580ab0d-f488-4191-9eea-b8d56bc21ded","Type":"ContainerStarted","Data":"85c9bec4e43bcd9ada796efec77f84abb91756fc6a076fbde8d0dac729b99cc4"} Sep 09 06:43:44.718165 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:44.718135 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-78b989bfc-cl4hl" Sep 09 06:43:50.764848 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:50.764802 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-4zgn6" event={"ID":"f1089629-f49d-4225-8f67-332ea17548ca","Type":"ContainerStarted","Data":"e0bd836f639a43de22e4743908e3d8f01b46aa7fb566431a9d1c477c82398268"} Sep 09 06:43:50.765362 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:50.765191 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-4zgn6" Sep 09 06:43:50.785666 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:50.785631 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-4zgn6" Sep 09 06:43:50.787284 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:50.787017 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-4zgn6" podStartSLOduration=1.226160526 podStartE2EDuration="17.786998633s" totalImagesPullingTime="16.560838107s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:43:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:43:50.785212306 +0000 UTC m=+328.681629727" watchObservedRunningTime="2026-09-09 06:43:50.786998633 +0000 UTC m=+328.683416055" Sep 09 06:43:54.788096 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:54.788036 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54b48b5d89-5qhd2" event={"ID":"7580ab0d-f488-4191-9eea-b8d56bc21ded","Type":"ContainerStarted","Data":"563edecfda57c6dff6bbdd129f2651ea290a5b90baad279ae85007e0e0f6d1e4"} Sep 09 06:43:54.814977 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:54.814926 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-54b48b5d89-5qhd2" podStartSLOduration=2.292027629 podStartE2EDuration="12.814911014s" totalImagesPullingTime="10.522883385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:43:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:43:54.81395528 +0000 UTC m=+332.710372703" watchObservedRunningTime="2026-09-09 06:43:54.814911014 +0000 UTC m=+332.711328434" Sep 09 06:43:56.799222 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:56.799187 2559 generic.go:356] "Generic (PLEG): container finished" podID="dbf282a5-cce3-489a-93df-44e7b10ef6fd" containerID="fc91b30b55c53f6083eed212946de3de2f4fea186c7f2ea261acff805aba15cb" exitCode=0 Sep 09 06:43:56.799704 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:56.799260 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-sqtkx" event={"ID":"dbf282a5-cce3-489a-93df-44e7b10ef6fd","Type":"ContainerDied","Data":"fc91b30b55c53f6083eed212946de3de2f4fea186c7f2ea261acff805aba15cb"} Sep 09 06:43:56.799704 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:56.799646 2559 scope.go:117] "RemoveContainer" containerID="fc91b30b55c53f6083eed212946de3de2f4fea186c7f2ea261acff805aba15cb" Sep 09 06:43:57.805848 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:43:57.805811 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-sqtkx" event={"ID":"dbf282a5-cce3-489a-93df-44e7b10ef6fd","Type":"ContainerStarted","Data":"5626447766d618b61c8d9b25324515cba56d5240eaee07cada8dcb360bb67ac1"} Sep 09 06:44:02.731100 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:02.731047 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-798d7b96c-6srkk" podUID="fe89f009-e9d3-404a-b39a-895d216f3c22" containerName="registry" containerID="cri-o://3e0e845c6775477961f2d1ef0f3488603ce73286bd8fc77dfa11ae217b3d51fa" gracePeriod=30 Sep 09 06:44:02.821213 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:02.821181 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54b48b5d89-5qhd2"] Sep 09 06:44:02.915283 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:02.915251 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-54b48b5d89-5qhd2" Sep 09 06:44:03.006916 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:03.006846 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:44:03.109583 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:03.109552 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jswlk\" (UniqueName: \"kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-kube-api-access-jswlk\") pod \"fe89f009-e9d3-404a-b39a-895d216f3c22\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " Sep 09 06:44:03.109583 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:03.109589 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fe89f009-e9d3-404a-b39a-895d216f3c22-image-registry-private-configuration\") pod \"fe89f009-e9d3-404a-b39a-895d216f3c22\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " Sep 09 06:44:03.109820 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:03.109617 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-bound-sa-token\") pod \"fe89f009-e9d3-404a-b39a-895d216f3c22\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " Sep 09 06:44:03.109820 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:03.109647 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-certificates\") pod \"fe89f009-e9d3-404a-b39a-895d216f3c22\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " Sep 09 06:44:03.109820 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:03.109795 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fe89f009-e9d3-404a-b39a-895d216f3c22-installation-pull-secrets\") pod \"fe89f009-e9d3-404a-b39a-895d216f3c22\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " Sep 09 06:44:03.109971 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:03.109861 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fe89f009-e9d3-404a-b39a-895d216f3c22-ca-trust-extracted\") pod \"fe89f009-e9d3-404a-b39a-895d216f3c22\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " Sep 09 06:44:03.109971 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:03.109892 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fe89f009-e9d3-404a-b39a-895d216f3c22-trusted-ca\") pod \"fe89f009-e9d3-404a-b39a-895d216f3c22\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " Sep 09 06:44:03.109971 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:03.109925 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-tls\") pod \"fe89f009-e9d3-404a-b39a-895d216f3c22\" (UID: \"fe89f009-e9d3-404a-b39a-895d216f3c22\") " Sep 09 06:44:03.110163 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:03.110124 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "fe89f009-e9d3-404a-b39a-895d216f3c22" (UID: "fe89f009-e9d3-404a-b39a-895d216f3c22"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 06:44:03.110258 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:03.110239 2559 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-certificates\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:44:03.110501 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:03.110467 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe89f009-e9d3-404a-b39a-895d216f3c22-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "fe89f009-e9d3-404a-b39a-895d216f3c22" (UID: "fe89f009-e9d3-404a-b39a-895d216f3c22"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 06:44:03.112330 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:03.112305 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-kube-api-access-jswlk" (OuterVolumeSpecName: "kube-api-access-jswlk") pod "fe89f009-e9d3-404a-b39a-895d216f3c22" (UID: "fe89f009-e9d3-404a-b39a-895d216f3c22"). InnerVolumeSpecName "kube-api-access-jswlk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 06:44:03.112441 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:03.112310 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "fe89f009-e9d3-404a-b39a-895d216f3c22" (UID: "fe89f009-e9d3-404a-b39a-895d216f3c22"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 06:44:03.112441 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:03.112340 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe89f009-e9d3-404a-b39a-895d216f3c22-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "fe89f009-e9d3-404a-b39a-895d216f3c22" (UID: "fe89f009-e9d3-404a-b39a-895d216f3c22"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 06:44:03.112529 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:03.112460 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "fe89f009-e9d3-404a-b39a-895d216f3c22" (UID: "fe89f009-e9d3-404a-b39a-895d216f3c22"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 06:44:03.112721 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:03.112705 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe89f009-e9d3-404a-b39a-895d216f3c22-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "fe89f009-e9d3-404a-b39a-895d216f3c22" (UID: "fe89f009-e9d3-404a-b39a-895d216f3c22"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 06:44:03.118277 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:03.118250 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe89f009-e9d3-404a-b39a-895d216f3c22-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "fe89f009-e9d3-404a-b39a-895d216f3c22" (UID: "fe89f009-e9d3-404a-b39a-895d216f3c22"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 06:44:03.211320 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:03.211291 2559 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fe89f009-e9d3-404a-b39a-895d216f3c22-installation-pull-secrets\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:44:03.211320 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:03.211320 2559 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fe89f009-e9d3-404a-b39a-895d216f3c22-ca-trust-extracted\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:44:03.211487 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:03.211333 2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fe89f009-e9d3-404a-b39a-895d216f3c22-trusted-ca\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:44:03.211487 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:03.211346 2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-registry-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:44:03.211487 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:03.211356 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jswlk\" (UniqueName: \"kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-kube-api-access-jswlk\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:44:03.211487 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:03.211370 2559 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fe89f009-e9d3-404a-b39a-895d216f3c22-image-registry-private-configuration\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:44:03.211487 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:03.211384 2559 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fe89f009-e9d3-404a-b39a-895d216f3c22-bound-sa-token\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:44:03.831135 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:03.831099 2559 generic.go:356] "Generic (PLEG): container finished" podID="fe89f009-e9d3-404a-b39a-895d216f3c22" containerID="3e0e845c6775477961f2d1ef0f3488603ce73286bd8fc77dfa11ae217b3d51fa" exitCode=0 Sep 09 06:44:03.831554 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:03.831160 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-798d7b96c-6srkk" Sep 09 06:44:03.831554 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:03.831186 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-798d7b96c-6srkk" event={"ID":"fe89f009-e9d3-404a-b39a-895d216f3c22","Type":"ContainerDied","Data":"3e0e845c6775477961f2d1ef0f3488603ce73286bd8fc77dfa11ae217b3d51fa"} Sep 09 06:44:03.831554 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:03.831230 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-798d7b96c-6srkk" event={"ID":"fe89f009-e9d3-404a-b39a-895d216f3c22","Type":"ContainerDied","Data":"060e3906253878f64cd53a98cc0ba54d3a8625e8dff79195da16b9793fb7cfde"} Sep 09 06:44:03.831554 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:03.831248 2559 scope.go:117] "RemoveContainer" containerID="3e0e845c6775477961f2d1ef0f3488603ce73286bd8fc77dfa11ae217b3d51fa" Sep 09 06:44:03.840171 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:03.840146 2559 scope.go:117] "RemoveContainer" containerID="3e0e845c6775477961f2d1ef0f3488603ce73286bd8fc77dfa11ae217b3d51fa" Sep 09 06:44:03.840438 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:44:03.840421 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e0e845c6775477961f2d1ef0f3488603ce73286bd8fc77dfa11ae217b3d51fa\": container with ID starting with 3e0e845c6775477961f2d1ef0f3488603ce73286bd8fc77dfa11ae217b3d51fa not found: ID does not exist" containerID="3e0e845c6775477961f2d1ef0f3488603ce73286bd8fc77dfa11ae217b3d51fa" Sep 09 06:44:03.840509 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:03.840445 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e0e845c6775477961f2d1ef0f3488603ce73286bd8fc77dfa11ae217b3d51fa"} err="failed to get container status \"3e0e845c6775477961f2d1ef0f3488603ce73286bd8fc77dfa11ae217b3d51fa\": rpc error: code = NotFound desc = could not find container \"3e0e845c6775477961f2d1ef0f3488603ce73286bd8fc77dfa11ae217b3d51fa\": container with ID starting with 3e0e845c6775477961f2d1ef0f3488603ce73286bd8fc77dfa11ae217b3d51fa not found: ID does not exist" Sep 09 06:44:03.854452 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:03.854429 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-798d7b96c-6srkk"] Sep 09 06:44:03.863114 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:03.862886 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-798d7b96c-6srkk"] Sep 09 06:44:04.701752 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:04.701715 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe89f009-e9d3-404a-b39a-895d216f3c22" path="/var/lib/kubelet/pods/fe89f009-e9d3-404a-b39a-895d216f3c22/volumes" Sep 09 06:44:11.869049 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:11.869014 2559 generic.go:356] "Generic (PLEG): container finished" podID="781bd79e-3b6d-493f-a5e1-aaeecb110ffd" containerID="481dc43b5eb08cfda7e716390e949c5d9091155a28a6762fdb3cbca38f23d669" exitCode=0 Sep 09 06:44:11.869585 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:11.869094 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-lw25n" event={"ID":"781bd79e-3b6d-493f-a5e1-aaeecb110ffd","Type":"ContainerDied","Data":"481dc43b5eb08cfda7e716390e949c5d9091155a28a6762fdb3cbca38f23d669"} Sep 09 06:44:11.869585 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:11.869564 2559 scope.go:117] "RemoveContainer" containerID="481dc43b5eb08cfda7e716390e949c5d9091155a28a6762fdb3cbca38f23d669" Sep 09 06:44:12.874668 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:12.874636 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-lw25n" event={"ID":"781bd79e-3b6d-493f-a5e1-aaeecb110ffd","Type":"ContainerStarted","Data":"21174c208cac17ea95bb12304a7b62ce662ee228d618169696c6c03fab269ade"} Sep 09 06:44:27.842264 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:27.842205 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-54b48b5d89-5qhd2" podUID="7580ab0d-f488-4191-9eea-b8d56bc21ded" containerName="console" containerID="cri-o://563edecfda57c6dff6bbdd129f2651ea290a5b90baad279ae85007e0e0f6d1e4" gracePeriod=15 Sep 09 06:44:28.100195 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:28.100124 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54b48b5d89-5qhd2" Sep 09 06:44:28.219677 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:28.219639 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7580ab0d-f488-4191-9eea-b8d56bc21ded-service-ca\") pod \"7580ab0d-f488-4191-9eea-b8d56bc21ded\" (UID: \"7580ab0d-f488-4191-9eea-b8d56bc21ded\") " Sep 09 06:44:28.219677 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:28.219683 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hmghn\" (UniqueName: \"kubernetes.io/projected/7580ab0d-f488-4191-9eea-b8d56bc21ded-kube-api-access-hmghn\") pod \"7580ab0d-f488-4191-9eea-b8d56bc21ded\" (UID: \"7580ab0d-f488-4191-9eea-b8d56bc21ded\") " Sep 09 06:44:28.219893 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:28.219715 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7580ab0d-f488-4191-9eea-b8d56bc21ded-console-oauth-config\") pod \"7580ab0d-f488-4191-9eea-b8d56bc21ded\" (UID: \"7580ab0d-f488-4191-9eea-b8d56bc21ded\") " Sep 09 06:44:28.219893 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:28.219837 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7580ab0d-f488-4191-9eea-b8d56bc21ded-console-config\") pod \"7580ab0d-f488-4191-9eea-b8d56bc21ded\" (UID: \"7580ab0d-f488-4191-9eea-b8d56bc21ded\") " Sep 09 06:44:28.219893 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:28.219887 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7580ab0d-f488-4191-9eea-b8d56bc21ded-console-serving-cert\") pod \"7580ab0d-f488-4191-9eea-b8d56bc21ded\" (UID: \"7580ab0d-f488-4191-9eea-b8d56bc21ded\") " Sep 09 06:44:28.220036 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:28.219928 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7580ab0d-f488-4191-9eea-b8d56bc21ded-oauth-serving-cert\") pod \"7580ab0d-f488-4191-9eea-b8d56bc21ded\" (UID: \"7580ab0d-f488-4191-9eea-b8d56bc21ded\") " Sep 09 06:44:28.220173 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:28.220135 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7580ab0d-f488-4191-9eea-b8d56bc21ded-service-ca" (OuterVolumeSpecName: "service-ca") pod "7580ab0d-f488-4191-9eea-b8d56bc21ded" (UID: "7580ab0d-f488-4191-9eea-b8d56bc21ded"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 06:44:28.220359 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:28.220318 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7580ab0d-f488-4191-9eea-b8d56bc21ded-console-config" (OuterVolumeSpecName: "console-config") pod "7580ab0d-f488-4191-9eea-b8d56bc21ded" (UID: "7580ab0d-f488-4191-9eea-b8d56bc21ded"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 06:44:28.220359 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:28.220344 2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7580ab0d-f488-4191-9eea-b8d56bc21ded-service-ca\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:44:28.220461 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:28.220377 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7580ab0d-f488-4191-9eea-b8d56bc21ded-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7580ab0d-f488-4191-9eea-b8d56bc21ded" (UID: "7580ab0d-f488-4191-9eea-b8d56bc21ded"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 06:44:28.221927 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:28.221908 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7580ab0d-f488-4191-9eea-b8d56bc21ded-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7580ab0d-f488-4191-9eea-b8d56bc21ded" (UID: "7580ab0d-f488-4191-9eea-b8d56bc21ded"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 06:44:28.222330 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:28.222312 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7580ab0d-f488-4191-9eea-b8d56bc21ded-kube-api-access-hmghn" (OuterVolumeSpecName: "kube-api-access-hmghn") pod "7580ab0d-f488-4191-9eea-b8d56bc21ded" (UID: "7580ab0d-f488-4191-9eea-b8d56bc21ded"). InnerVolumeSpecName "kube-api-access-hmghn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 06:44:28.222330 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:28.222315 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7580ab0d-f488-4191-9eea-b8d56bc21ded-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7580ab0d-f488-4191-9eea-b8d56bc21ded" (UID: "7580ab0d-f488-4191-9eea-b8d56bc21ded"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 06:44:28.321368 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:28.321336 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmghn\" (UniqueName: \"kubernetes.io/projected/7580ab0d-f488-4191-9eea-b8d56bc21ded-kube-api-access-hmghn\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:44:28.321368 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:28.321362 2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7580ab0d-f488-4191-9eea-b8d56bc21ded-console-oauth-config\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:44:28.321516 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:28.321375 2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7580ab0d-f488-4191-9eea-b8d56bc21ded-console-config\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:44:28.321516 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:28.321389 2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7580ab0d-f488-4191-9eea-b8d56bc21ded-console-serving-cert\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:44:28.321516 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:28.321401 2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7580ab0d-f488-4191-9eea-b8d56bc21ded-oauth-serving-cert\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:44:28.939217 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:28.939185 2559 generic.go:356] "Generic (PLEG): container finished" podID="7580ab0d-f488-4191-9eea-b8d56bc21ded" containerID="563edecfda57c6dff6bbdd129f2651ea290a5b90baad279ae85007e0e0f6d1e4" exitCode=0 Sep 09 06:44:28.939601 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:28.939231 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54b48b5d89-5qhd2" Sep 09 06:44:28.939601 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:28.939232 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54b48b5d89-5qhd2" event={"ID":"7580ab0d-f488-4191-9eea-b8d56bc21ded","Type":"ContainerDied","Data":"563edecfda57c6dff6bbdd129f2651ea290a5b90baad279ae85007e0e0f6d1e4"} Sep 09 06:44:28.939601 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:28.939363 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54b48b5d89-5qhd2" event={"ID":"7580ab0d-f488-4191-9eea-b8d56bc21ded","Type":"ContainerDied","Data":"85c9bec4e43bcd9ada796efec77f84abb91756fc6a076fbde8d0dac729b99cc4"} Sep 09 06:44:28.939601 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:28.939394 2559 scope.go:117] "RemoveContainer" containerID="563edecfda57c6dff6bbdd129f2651ea290a5b90baad279ae85007e0e0f6d1e4" Sep 09 06:44:28.947349 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:28.947328 2559 scope.go:117] "RemoveContainer" containerID="563edecfda57c6dff6bbdd129f2651ea290a5b90baad279ae85007e0e0f6d1e4" Sep 09 06:44:28.947626 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:44:28.947601 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"563edecfda57c6dff6bbdd129f2651ea290a5b90baad279ae85007e0e0f6d1e4\": container with ID starting with 563edecfda57c6dff6bbdd129f2651ea290a5b90baad279ae85007e0e0f6d1e4 not found: ID does not exist" containerID="563edecfda57c6dff6bbdd129f2651ea290a5b90baad279ae85007e0e0f6d1e4" Sep 09 06:44:28.947695 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:28.947634 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"563edecfda57c6dff6bbdd129f2651ea290a5b90baad279ae85007e0e0f6d1e4"} err="failed to get container status \"563edecfda57c6dff6bbdd129f2651ea290a5b90baad279ae85007e0e0f6d1e4\": rpc error: code = NotFound desc = could not find container \"563edecfda57c6dff6bbdd129f2651ea290a5b90baad279ae85007e0e0f6d1e4\": container with ID starting with 563edecfda57c6dff6bbdd129f2651ea290a5b90baad279ae85007e0e0f6d1e4 not found: ID does not exist" Sep 09 06:44:28.959216 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:28.959168 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54b48b5d89-5qhd2"] Sep 09 06:44:28.965856 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:28.965837 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-54b48b5d89-5qhd2"] Sep 09 06:44:30.701410 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:30.701378 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7580ab0d-f488-4191-9eea-b8d56bc21ded" path="/var/lib/kubelet/pods/7580ab0d-f488-4191-9eea-b8d56bc21ded/volumes" Sep 09 06:44:36.385740 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:36.385697 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/336552c8-ca0f-434a-8353-bc61ea118fa5-metrics-certs\") pod \"network-metrics-daemon-pkv72\" (UID: \"336552c8-ca0f-434a-8353-bc61ea118fa5\") " pod="openshift-multus/network-metrics-daemon-pkv72" Sep 09 06:44:36.388012 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:36.387988 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/336552c8-ca0f-434a-8353-bc61ea118fa5-metrics-certs\") pod \"network-metrics-daemon-pkv72\" (UID: \"336552c8-ca0f-434a-8353-bc61ea118fa5\") " pod="openshift-multus/network-metrics-daemon-pkv72" Sep 09 06:44:36.599186 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:36.599143 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tmq86\"" Sep 09 06:44:36.607995 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:36.607969 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkv72" Sep 09 06:44:36.904958 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:36.904890 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pkv72"] Sep 09 06:44:36.907844 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:44:36.907819 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod336552c8_ca0f_434a_8353_bc61ea118fa5.slice/crio-6b49e49d82ebda85c17d2f66485b7de9d89789c4039488e762dd0eef0707e45a WatchSource:0}: Error finding container 6b49e49d82ebda85c17d2f66485b7de9d89789c4039488e762dd0eef0707e45a: Status 404 returned error can't find the container with id 6b49e49d82ebda85c17d2f66485b7de9d89789c4039488e762dd0eef0707e45a Sep 09 06:44:36.973578 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:36.973546 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pkv72" event={"ID":"336552c8-ca0f-434a-8353-bc61ea118fa5","Type":"ContainerStarted","Data":"6b49e49d82ebda85c17d2f66485b7de9d89789c4039488e762dd0eef0707e45a"} Sep 09 06:44:38.983919 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:38.983880 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pkv72" event={"ID":"336552c8-ca0f-434a-8353-bc61ea118fa5","Type":"ContainerStarted","Data":"7d68a7d2fa33f49aca90e5191c5792b18624959b619efb5b5cc4c8d0f1fd3560"} Sep 09 06:44:38.984430 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:38.983943 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pkv72" event={"ID":"336552c8-ca0f-434a-8353-bc61ea118fa5","Type":"ContainerStarted","Data":"b44abfc82d41856fc756755f4ea937789a8d4d8416a5549624cc911b1b089454"} Sep 09 06:44:39.000580 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:39.000533 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-pkv72" podStartSLOduration=375.516555154 podStartE2EDuration="6m17.000504846s" totalImagesPullingTime="1.483949692s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:38:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:44:38.998799252 +0000 UTC m=+376.895216687" watchObservedRunningTime="2026-09-09 06:44:39.000504846 +0000 UTC m=+376.896922267" Sep 09 06:44:50.048554 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:50.048517 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 06:44:50.049159 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:50.048860 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="73fabe44-d13f-4841-8196-f0080f9d24e7" containerName="alertmanager" containerID="cri-o://c7bc791eae25fd65c248934ca459991af30fb3de2d9fa6f03c10fb4c265aad9e" gracePeriod=120 Sep 09 06:44:50.049159 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:50.048885 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="73fabe44-d13f-4841-8196-f0080f9d24e7" containerName="kube-rbac-proxy-metric" containerID="cri-o://e4d75f04171b0e9f4697e5e1e48aed8f62780b4a3a4a15a970ee7aa9d7c1d6d8" gracePeriod=120 Sep 09 06:44:50.049159 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:50.048912 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="73fabe44-d13f-4841-8196-f0080f9d24e7" containerName="kube-rbac-proxy" containerID="cri-o://8841b40bf8c2f336cf932c0d3b8d8a9bfdb7c0a36289552a218e71d8ed56a900" gracePeriod=120 Sep 09 06:44:50.049159 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:50.048941 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="73fabe44-d13f-4841-8196-f0080f9d24e7" containerName="prom-label-proxy" containerID="cri-o://8096172dd8bb7806ba753d2550c562a904e31e2bc48661fe4ffea37c532bb645" gracePeriod=120 Sep 09 06:44:50.049159 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:50.048899 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="73fabe44-d13f-4841-8196-f0080f9d24e7" containerName="kube-rbac-proxy-web" containerID="cri-o://8d57f23a0f1a81fbe6cc3e89d2040e38766e467e321966da0ff99c05c084ea12" gracePeriod=120 Sep 09 06:44:50.049159 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:50.049022 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="73fabe44-d13f-4841-8196-f0080f9d24e7" containerName="config-reloader" containerID="cri-o://ca80b536126fb47e88260039ec65b485c620bf172b32afc31718e8d38e0988f1" gracePeriod=120 Sep 09 06:44:51.034856 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.034820 2559 generic.go:356] "Generic (PLEG): container finished" podID="73fabe44-d13f-4841-8196-f0080f9d24e7" containerID="8096172dd8bb7806ba753d2550c562a904e31e2bc48661fe4ffea37c532bb645" exitCode=0 Sep 09 06:44:51.034856 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.034848 2559 generic.go:356] "Generic (PLEG): container finished" podID="73fabe44-d13f-4841-8196-f0080f9d24e7" containerID="8841b40bf8c2f336cf932c0d3b8d8a9bfdb7c0a36289552a218e71d8ed56a900" exitCode=0 Sep 09 06:44:51.034856 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.034856 2559 generic.go:356] "Generic (PLEG): container finished" podID="73fabe44-d13f-4841-8196-f0080f9d24e7" containerID="ca80b536126fb47e88260039ec65b485c620bf172b32afc31718e8d38e0988f1" exitCode=0 Sep 09 06:44:51.034856 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.034862 2559 generic.go:356] "Generic (PLEG): container finished" podID="73fabe44-d13f-4841-8196-f0080f9d24e7" containerID="c7bc791eae25fd65c248934ca459991af30fb3de2d9fa6f03c10fb4c265aad9e" exitCode=0 Sep 09 06:44:51.035202 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.034892 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"73fabe44-d13f-4841-8196-f0080f9d24e7","Type":"ContainerDied","Data":"8096172dd8bb7806ba753d2550c562a904e31e2bc48661fe4ffea37c532bb645"} Sep 09 06:44:51.035202 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.034974 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"73fabe44-d13f-4841-8196-f0080f9d24e7","Type":"ContainerDied","Data":"8841b40bf8c2f336cf932c0d3b8d8a9bfdb7c0a36289552a218e71d8ed56a900"} Sep 09 06:44:51.035202 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.035015 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"73fabe44-d13f-4841-8196-f0080f9d24e7","Type":"ContainerDied","Data":"ca80b536126fb47e88260039ec65b485c620bf172b32afc31718e8d38e0988f1"} Sep 09 06:44:51.035202 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.035068 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"73fabe44-d13f-4841-8196-f0080f9d24e7","Type":"ContainerDied","Data":"c7bc791eae25fd65c248934ca459991af30fb3de2d9fa6f03c10fb4c265aad9e"} Sep 09 06:44:51.294387 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.294324 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:51.406349 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.406319 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/73fabe44-d13f-4841-8196-f0080f9d24e7-tls-assets\") pod \"73fabe44-d13f-4841-8196-f0080f9d24e7\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " Sep 09 06:44:51.406524 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.406362 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/73fabe44-d13f-4841-8196-f0080f9d24e7-alertmanager-main-db\") pod \"73fabe44-d13f-4841-8196-f0080f9d24e7\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " Sep 09 06:44:51.406524 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.406390 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m47p9\" (UniqueName: \"kubernetes.io/projected/73fabe44-d13f-4841-8196-f0080f9d24e7-kube-api-access-m47p9\") pod \"73fabe44-d13f-4841-8196-f0080f9d24e7\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " Sep 09 06:44:51.406524 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.406421 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73fabe44-d13f-4841-8196-f0080f9d24e7-metrics-client-ca\") pod \"73fabe44-d13f-4841-8196-f0080f9d24e7\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " Sep 09 06:44:51.406688 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.406590 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/73fabe44-d13f-4841-8196-f0080f9d24e7-config-out\") pod \"73fabe44-d13f-4841-8196-f0080f9d24e7\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " Sep 09 06:44:51.406688 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.406644 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73fabe44-d13f-4841-8196-f0080f9d24e7-alertmanager-trusted-ca-bundle\") pod \"73fabe44-d13f-4841-8196-f0080f9d24e7\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " Sep 09 06:44:51.406786 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.406692 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-secret-alertmanager-kube-rbac-proxy-web\") pod \"73fabe44-d13f-4841-8196-f0080f9d24e7\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " Sep 09 06:44:51.406786 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.406719 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-web-config\") pod \"73fabe44-d13f-4841-8196-f0080f9d24e7\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " Sep 09 06:44:51.406786 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.406766 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-config-volume\") pod \"73fabe44-d13f-4841-8196-f0080f9d24e7\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " Sep 09 06:44:51.406940 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.406793 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-secret-alertmanager-main-tls\") pod \"73fabe44-d13f-4841-8196-f0080f9d24e7\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " Sep 09 06:44:51.406940 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.406816 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73fabe44-d13f-4841-8196-f0080f9d24e7-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "73fabe44-d13f-4841-8196-f0080f9d24e7" (UID: "73fabe44-d13f-4841-8196-f0080f9d24e7"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 06:44:51.406940 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.406824 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-secret-alertmanager-kube-rbac-proxy\") pod \"73fabe44-d13f-4841-8196-f0080f9d24e7\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " Sep 09 06:44:51.406940 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.406854 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-cluster-tls-config\") pod \"73fabe44-d13f-4841-8196-f0080f9d24e7\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " Sep 09 06:44:51.406940 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.406892 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"73fabe44-d13f-4841-8196-f0080f9d24e7\" (UID: \"73fabe44-d13f-4841-8196-f0080f9d24e7\") " Sep 09 06:44:51.407221 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.407086 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73fabe44-d13f-4841-8196-f0080f9d24e7-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "73fabe44-d13f-4841-8196-f0080f9d24e7" (UID: "73fabe44-d13f-4841-8196-f0080f9d24e7"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 06:44:51.407221 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.407187 2559 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73fabe44-d13f-4841-8196-f0080f9d24e7-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:44:51.407221 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.407207 2559 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73fabe44-d13f-4841-8196-f0080f9d24e7-metrics-client-ca\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:44:51.407613 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.407465 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73fabe44-d13f-4841-8196-f0080f9d24e7-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "73fabe44-d13f-4841-8196-f0080f9d24e7" (UID: "73fabe44-d13f-4841-8196-f0080f9d24e7"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 06:44:51.409270 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.409229 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73fabe44-d13f-4841-8196-f0080f9d24e7-kube-api-access-m47p9" (OuterVolumeSpecName: "kube-api-access-m47p9") pod "73fabe44-d13f-4841-8196-f0080f9d24e7" (UID: "73fabe44-d13f-4841-8196-f0080f9d24e7"). InnerVolumeSpecName "kube-api-access-m47p9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 06:44:51.409566 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.409526 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73fabe44-d13f-4841-8196-f0080f9d24e7-config-out" (OuterVolumeSpecName: "config-out") pod "73fabe44-d13f-4841-8196-f0080f9d24e7" (UID: "73fabe44-d13f-4841-8196-f0080f9d24e7"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 06:44:51.409744 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.409714 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73fabe44-d13f-4841-8196-f0080f9d24e7-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "73fabe44-d13f-4841-8196-f0080f9d24e7" (UID: "73fabe44-d13f-4841-8196-f0080f9d24e7"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 06:44:51.409808 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.409743 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-config-volume" (OuterVolumeSpecName: "config-volume") pod "73fabe44-d13f-4841-8196-f0080f9d24e7" (UID: "73fabe44-d13f-4841-8196-f0080f9d24e7"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 06:44:51.409985 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.409957 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "73fabe44-d13f-4841-8196-f0080f9d24e7" (UID: "73fabe44-d13f-4841-8196-f0080f9d24e7"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 06:44:51.410479 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.410455 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "73fabe44-d13f-4841-8196-f0080f9d24e7" (UID: "73fabe44-d13f-4841-8196-f0080f9d24e7"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 06:44:51.410717 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.410686 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "73fabe44-d13f-4841-8196-f0080f9d24e7" (UID: "73fabe44-d13f-4841-8196-f0080f9d24e7"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 06:44:51.411024 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.410996 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "73fabe44-d13f-4841-8196-f0080f9d24e7" (UID: "73fabe44-d13f-4841-8196-f0080f9d24e7"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 06:44:51.414156 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.413967 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "73fabe44-d13f-4841-8196-f0080f9d24e7" (UID: "73fabe44-d13f-4841-8196-f0080f9d24e7"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 06:44:51.420588 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.420567 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-web-config" (OuterVolumeSpecName: "web-config") pod "73fabe44-d13f-4841-8196-f0080f9d24e7" (UID: "73fabe44-d13f-4841-8196-f0080f9d24e7"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 06:44:51.508247 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.508221 2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:44:51.508247 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.508243 2559 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-web-config\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:44:51.508385 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.508253 2559 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-config-volume\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:44:51.508385 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.508261 2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-secret-alertmanager-main-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:44:51.508385 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.508270 2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:44:51.508385 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.508278 2559 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-cluster-tls-config\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:44:51.508385 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.508287 2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/73fabe44-d13f-4841-8196-f0080f9d24e7-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:44:51.508385 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.508296 2559 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/73fabe44-d13f-4841-8196-f0080f9d24e7-tls-assets\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:44:51.508385 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.508305 2559 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/73fabe44-d13f-4841-8196-f0080f9d24e7-alertmanager-main-db\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:44:51.508385 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.508313 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m47p9\" (UniqueName: \"kubernetes.io/projected/73fabe44-d13f-4841-8196-f0080f9d24e7-kube-api-access-m47p9\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:44:51.508385 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:51.508321 2559 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/73fabe44-d13f-4841-8196-f0080f9d24e7-config-out\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:44:52.042374 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.042343 2559 generic.go:356] "Generic (PLEG): container finished" podID="73fabe44-d13f-4841-8196-f0080f9d24e7" containerID="e4d75f04171b0e9f4697e5e1e48aed8f62780b4a3a4a15a970ee7aa9d7c1d6d8" exitCode=0 Sep 09 06:44:52.042374 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.042367 2559 generic.go:356] "Generic (PLEG): container finished" podID="73fabe44-d13f-4841-8196-f0080f9d24e7" containerID="8d57f23a0f1a81fbe6cc3e89d2040e38766e467e321966da0ff99c05c084ea12" exitCode=0 Sep 09 06:44:52.042576 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.042417 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"73fabe44-d13f-4841-8196-f0080f9d24e7","Type":"ContainerDied","Data":"e4d75f04171b0e9f4697e5e1e48aed8f62780b4a3a4a15a970ee7aa9d7c1d6d8"} Sep 09 06:44:52.042576 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.042423 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.042576 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.042470 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"73fabe44-d13f-4841-8196-f0080f9d24e7","Type":"ContainerDied","Data":"8d57f23a0f1a81fbe6cc3e89d2040e38766e467e321966da0ff99c05c084ea12"} Sep 09 06:44:52.042576 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.042498 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"73fabe44-d13f-4841-8196-f0080f9d24e7","Type":"ContainerDied","Data":"3681098c91c6489758e6fee3ad31c01abd6f9cd99e4bdc82bc4fa2592f29ca1a"} Sep 09 06:44:52.042576 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.042528 2559 scope.go:117] "RemoveContainer" containerID="8096172dd8bb7806ba753d2550c562a904e31e2bc48661fe4ffea37c532bb645" Sep 09 06:44:52.050342 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.050309 2559 scope.go:117] "RemoveContainer" containerID="e4d75f04171b0e9f4697e5e1e48aed8f62780b4a3a4a15a970ee7aa9d7c1d6d8" Sep 09 06:44:52.059667 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.059648 2559 scope.go:117] "RemoveContainer" containerID="8841b40bf8c2f336cf932c0d3b8d8a9bfdb7c0a36289552a218e71d8ed56a900" Sep 09 06:44:52.066632 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.066526 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 06:44:52.066685 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.066658 2559 scope.go:117] "RemoveContainer" containerID="8d57f23a0f1a81fbe6cc3e89d2040e38766e467e321966da0ff99c05c084ea12" Sep 09 06:44:52.073093 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.073069 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 06:44:52.073661 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.073643 2559 scope.go:117] "RemoveContainer" containerID="ca80b536126fb47e88260039ec65b485c620bf172b32afc31718e8d38e0988f1" Sep 09 06:44:52.080319 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.080300 2559 scope.go:117] "RemoveContainer" containerID="c7bc791eae25fd65c248934ca459991af30fb3de2d9fa6f03c10fb4c265aad9e" Sep 09 06:44:52.087213 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.087195 2559 scope.go:117] "RemoveContainer" containerID="3a430b79605ba0a8116bc672e81d1946fbb69ad45700702aa2b7eb8ed738a5c2" Sep 09 06:44:52.094124 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.094104 2559 scope.go:117] "RemoveContainer" containerID="8096172dd8bb7806ba753d2550c562a904e31e2bc48661fe4ffea37c532bb645" Sep 09 06:44:52.094411 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:44:52.094390 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8096172dd8bb7806ba753d2550c562a904e31e2bc48661fe4ffea37c532bb645\": container with ID starting with 8096172dd8bb7806ba753d2550c562a904e31e2bc48661fe4ffea37c532bb645 not found: ID does not exist" containerID="8096172dd8bb7806ba753d2550c562a904e31e2bc48661fe4ffea37c532bb645" Sep 09 06:44:52.094478 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.094419 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8096172dd8bb7806ba753d2550c562a904e31e2bc48661fe4ffea37c532bb645"} err="failed to get container status \"8096172dd8bb7806ba753d2550c562a904e31e2bc48661fe4ffea37c532bb645\": rpc error: code = NotFound desc = could not find container \"8096172dd8bb7806ba753d2550c562a904e31e2bc48661fe4ffea37c532bb645\": container with ID starting with 8096172dd8bb7806ba753d2550c562a904e31e2bc48661fe4ffea37c532bb645 not found: ID does not exist" Sep 09 06:44:52.094478 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.094439 2559 scope.go:117] "RemoveContainer" containerID="e4d75f04171b0e9f4697e5e1e48aed8f62780b4a3a4a15a970ee7aa9d7c1d6d8" Sep 09 06:44:52.094693 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:44:52.094679 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4d75f04171b0e9f4697e5e1e48aed8f62780b4a3a4a15a970ee7aa9d7c1d6d8\": container with ID starting with e4d75f04171b0e9f4697e5e1e48aed8f62780b4a3a4a15a970ee7aa9d7c1d6d8 not found: ID does not exist" containerID="e4d75f04171b0e9f4697e5e1e48aed8f62780b4a3a4a15a970ee7aa9d7c1d6d8" Sep 09 06:44:52.094733 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.094699 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4d75f04171b0e9f4697e5e1e48aed8f62780b4a3a4a15a970ee7aa9d7c1d6d8"} err="failed to get container status \"e4d75f04171b0e9f4697e5e1e48aed8f62780b4a3a4a15a970ee7aa9d7c1d6d8\": rpc error: code = NotFound desc = could not find container \"e4d75f04171b0e9f4697e5e1e48aed8f62780b4a3a4a15a970ee7aa9d7c1d6d8\": container with ID starting with e4d75f04171b0e9f4697e5e1e48aed8f62780b4a3a4a15a970ee7aa9d7c1d6d8 not found: ID does not exist" Sep 09 06:44:52.094733 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.094711 2559 scope.go:117] "RemoveContainer" containerID="8841b40bf8c2f336cf932c0d3b8d8a9bfdb7c0a36289552a218e71d8ed56a900" Sep 09 06:44:52.094946 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:44:52.094928 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8841b40bf8c2f336cf932c0d3b8d8a9bfdb7c0a36289552a218e71d8ed56a900\": container with ID starting with 8841b40bf8c2f336cf932c0d3b8d8a9bfdb7c0a36289552a218e71d8ed56a900 not found: ID does not exist" containerID="8841b40bf8c2f336cf932c0d3b8d8a9bfdb7c0a36289552a218e71d8ed56a900" Sep 09 06:44:52.094996 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.094950 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8841b40bf8c2f336cf932c0d3b8d8a9bfdb7c0a36289552a218e71d8ed56a900"} err="failed to get container status \"8841b40bf8c2f336cf932c0d3b8d8a9bfdb7c0a36289552a218e71d8ed56a900\": rpc error: code = NotFound desc = could not find container \"8841b40bf8c2f336cf932c0d3b8d8a9bfdb7c0a36289552a218e71d8ed56a900\": container with ID starting with 8841b40bf8c2f336cf932c0d3b8d8a9bfdb7c0a36289552a218e71d8ed56a900 not found: ID does not exist" Sep 09 06:44:52.094996 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.094962 2559 scope.go:117] "RemoveContainer" containerID="8d57f23a0f1a81fbe6cc3e89d2040e38766e467e321966da0ff99c05c084ea12" Sep 09 06:44:52.095181 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:44:52.095165 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d57f23a0f1a81fbe6cc3e89d2040e38766e467e321966da0ff99c05c084ea12\": container with ID starting with 8d57f23a0f1a81fbe6cc3e89d2040e38766e467e321966da0ff99c05c084ea12 not found: ID does not exist" containerID="8d57f23a0f1a81fbe6cc3e89d2040e38766e467e321966da0ff99c05c084ea12" Sep 09 06:44:52.095233 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.095185 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d57f23a0f1a81fbe6cc3e89d2040e38766e467e321966da0ff99c05c084ea12"} err="failed to get container status \"8d57f23a0f1a81fbe6cc3e89d2040e38766e467e321966da0ff99c05c084ea12\": rpc error: code = NotFound desc = could not find container \"8d57f23a0f1a81fbe6cc3e89d2040e38766e467e321966da0ff99c05c084ea12\": container with ID starting with 8d57f23a0f1a81fbe6cc3e89d2040e38766e467e321966da0ff99c05c084ea12 not found: ID does not exist" Sep 09 06:44:52.095233 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.095199 2559 scope.go:117] "RemoveContainer" containerID="ca80b536126fb47e88260039ec65b485c620bf172b32afc31718e8d38e0988f1" Sep 09 06:44:52.095404 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:44:52.095386 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca80b536126fb47e88260039ec65b485c620bf172b32afc31718e8d38e0988f1\": container with ID starting with ca80b536126fb47e88260039ec65b485c620bf172b32afc31718e8d38e0988f1 not found: ID does not exist" containerID="ca80b536126fb47e88260039ec65b485c620bf172b32afc31718e8d38e0988f1" Sep 09 06:44:52.095439 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.095413 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca80b536126fb47e88260039ec65b485c620bf172b32afc31718e8d38e0988f1"} err="failed to get container status \"ca80b536126fb47e88260039ec65b485c620bf172b32afc31718e8d38e0988f1\": rpc error: code = NotFound desc = could not find container \"ca80b536126fb47e88260039ec65b485c620bf172b32afc31718e8d38e0988f1\": container with ID starting with ca80b536126fb47e88260039ec65b485c620bf172b32afc31718e8d38e0988f1 not found: ID does not exist" Sep 09 06:44:52.095439 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.095425 2559 scope.go:117] "RemoveContainer" containerID="c7bc791eae25fd65c248934ca459991af30fb3de2d9fa6f03c10fb4c265aad9e" Sep 09 06:44:52.095637 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:44:52.095616 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7bc791eae25fd65c248934ca459991af30fb3de2d9fa6f03c10fb4c265aad9e\": container with ID starting with c7bc791eae25fd65c248934ca459991af30fb3de2d9fa6f03c10fb4c265aad9e not found: ID does not exist" containerID="c7bc791eae25fd65c248934ca459991af30fb3de2d9fa6f03c10fb4c265aad9e" Sep 09 06:44:52.095670 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.095644 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7bc791eae25fd65c248934ca459991af30fb3de2d9fa6f03c10fb4c265aad9e"} err="failed to get container status \"c7bc791eae25fd65c248934ca459991af30fb3de2d9fa6f03c10fb4c265aad9e\": rpc error: code = NotFound desc = could not find container \"c7bc791eae25fd65c248934ca459991af30fb3de2d9fa6f03c10fb4c265aad9e\": container with ID starting with c7bc791eae25fd65c248934ca459991af30fb3de2d9fa6f03c10fb4c265aad9e not found: ID does not exist" Sep 09 06:44:52.095670 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.095660 2559 scope.go:117] "RemoveContainer" containerID="3a430b79605ba0a8116bc672e81d1946fbb69ad45700702aa2b7eb8ed738a5c2" Sep 09 06:44:52.095867 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:44:52.095848 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a430b79605ba0a8116bc672e81d1946fbb69ad45700702aa2b7eb8ed738a5c2\": container with ID starting with 3a430b79605ba0a8116bc672e81d1946fbb69ad45700702aa2b7eb8ed738a5c2 not found: ID does not exist" containerID="3a430b79605ba0a8116bc672e81d1946fbb69ad45700702aa2b7eb8ed738a5c2" Sep 09 06:44:52.095903 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.095870 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a430b79605ba0a8116bc672e81d1946fbb69ad45700702aa2b7eb8ed738a5c2"} err="failed to get container status \"3a430b79605ba0a8116bc672e81d1946fbb69ad45700702aa2b7eb8ed738a5c2\": rpc error: code = NotFound desc = could not find container \"3a430b79605ba0a8116bc672e81d1946fbb69ad45700702aa2b7eb8ed738a5c2\": container with ID starting with 3a430b79605ba0a8116bc672e81d1946fbb69ad45700702aa2b7eb8ed738a5c2 not found: ID does not exist" Sep 09 06:44:52.095903 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.095883 2559 scope.go:117] "RemoveContainer" containerID="8096172dd8bb7806ba753d2550c562a904e31e2bc48661fe4ffea37c532bb645" Sep 09 06:44:52.096090 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.096070 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8096172dd8bb7806ba753d2550c562a904e31e2bc48661fe4ffea37c532bb645"} err="failed to get container status \"8096172dd8bb7806ba753d2550c562a904e31e2bc48661fe4ffea37c532bb645\": rpc error: code = NotFound desc = could not find container \"8096172dd8bb7806ba753d2550c562a904e31e2bc48661fe4ffea37c532bb645\": container with ID starting with 8096172dd8bb7806ba753d2550c562a904e31e2bc48661fe4ffea37c532bb645 not found: ID does not exist" Sep 09 06:44:52.096090 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.096087 2559 scope.go:117] "RemoveContainer" containerID="e4d75f04171b0e9f4697e5e1e48aed8f62780b4a3a4a15a970ee7aa9d7c1d6d8" Sep 09 06:44:52.096311 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.096296 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4d75f04171b0e9f4697e5e1e48aed8f62780b4a3a4a15a970ee7aa9d7c1d6d8"} err="failed to get container status \"e4d75f04171b0e9f4697e5e1e48aed8f62780b4a3a4a15a970ee7aa9d7c1d6d8\": rpc error: code = NotFound desc = could not find container \"e4d75f04171b0e9f4697e5e1e48aed8f62780b4a3a4a15a970ee7aa9d7c1d6d8\": container with ID starting with e4d75f04171b0e9f4697e5e1e48aed8f62780b4a3a4a15a970ee7aa9d7c1d6d8 not found: ID does not exist" Sep 09 06:44:52.096311 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.096310 2559 scope.go:117] "RemoveContainer" containerID="8841b40bf8c2f336cf932c0d3b8d8a9bfdb7c0a36289552a218e71d8ed56a900" Sep 09 06:44:52.096535 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.096520 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8841b40bf8c2f336cf932c0d3b8d8a9bfdb7c0a36289552a218e71d8ed56a900"} err="failed to get container status \"8841b40bf8c2f336cf932c0d3b8d8a9bfdb7c0a36289552a218e71d8ed56a900\": rpc error: code = NotFound desc = could not find container \"8841b40bf8c2f336cf932c0d3b8d8a9bfdb7c0a36289552a218e71d8ed56a900\": container with ID starting with 8841b40bf8c2f336cf932c0d3b8d8a9bfdb7c0a36289552a218e71d8ed56a900 not found: ID does not exist" Sep 09 06:44:52.096577 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.096535 2559 scope.go:117] "RemoveContainer" containerID="8d57f23a0f1a81fbe6cc3e89d2040e38766e467e321966da0ff99c05c084ea12" Sep 09 06:44:52.096751 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.096734 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d57f23a0f1a81fbe6cc3e89d2040e38766e467e321966da0ff99c05c084ea12"} err="failed to get container status \"8d57f23a0f1a81fbe6cc3e89d2040e38766e467e321966da0ff99c05c084ea12\": rpc error: code = NotFound desc = could not find container \"8d57f23a0f1a81fbe6cc3e89d2040e38766e467e321966da0ff99c05c084ea12\": container with ID starting with 8d57f23a0f1a81fbe6cc3e89d2040e38766e467e321966da0ff99c05c084ea12 not found: ID does not exist" Sep 09 06:44:52.096751 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.096749 2559 scope.go:117] "RemoveContainer" containerID="ca80b536126fb47e88260039ec65b485c620bf172b32afc31718e8d38e0988f1" Sep 09 06:44:52.096980 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.096954 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca80b536126fb47e88260039ec65b485c620bf172b32afc31718e8d38e0988f1"} err="failed to get container status \"ca80b536126fb47e88260039ec65b485c620bf172b32afc31718e8d38e0988f1\": rpc error: code = NotFound desc = could not find container \"ca80b536126fb47e88260039ec65b485c620bf172b32afc31718e8d38e0988f1\": container with ID starting with ca80b536126fb47e88260039ec65b485c620bf172b32afc31718e8d38e0988f1 not found: ID does not exist" Sep 09 06:44:52.097025 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.096981 2559 scope.go:117] "RemoveContainer" containerID="c7bc791eae25fd65c248934ca459991af30fb3de2d9fa6f03c10fb4c265aad9e" Sep 09 06:44:52.097241 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.097225 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7bc791eae25fd65c248934ca459991af30fb3de2d9fa6f03c10fb4c265aad9e"} err="failed to get container status \"c7bc791eae25fd65c248934ca459991af30fb3de2d9fa6f03c10fb4c265aad9e\": rpc error: code = NotFound desc = could not find container \"c7bc791eae25fd65c248934ca459991af30fb3de2d9fa6f03c10fb4c265aad9e\": container with ID starting with c7bc791eae25fd65c248934ca459991af30fb3de2d9fa6f03c10fb4c265aad9e not found: ID does not exist" Sep 09 06:44:52.097291 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.097243 2559 scope.go:117] "RemoveContainer" containerID="3a430b79605ba0a8116bc672e81d1946fbb69ad45700702aa2b7eb8ed738a5c2" Sep 09 06:44:52.097509 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.097489 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a430b79605ba0a8116bc672e81d1946fbb69ad45700702aa2b7eb8ed738a5c2"} err="failed to get container status \"3a430b79605ba0a8116bc672e81d1946fbb69ad45700702aa2b7eb8ed738a5c2\": rpc error: code = NotFound desc = could not find container \"3a430b79605ba0a8116bc672e81d1946fbb69ad45700702aa2b7eb8ed738a5c2\": container with ID starting with 3a430b79605ba0a8116bc672e81d1946fbb69ad45700702aa2b7eb8ed738a5c2 not found: ID does not exist" Sep 09 06:44:52.101541 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.101521 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 06:44:52.101826 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.101815 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73fabe44-d13f-4841-8196-f0080f9d24e7" containerName="kube-rbac-proxy" Sep 09 06:44:52.101874 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.101827 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="73fabe44-d13f-4841-8196-f0080f9d24e7" containerName="kube-rbac-proxy" Sep 09 06:44:52.101874 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.101838 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73fabe44-d13f-4841-8196-f0080f9d24e7" containerName="prom-label-proxy" Sep 09 06:44:52.101874 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.101844 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="73fabe44-d13f-4841-8196-f0080f9d24e7" containerName="prom-label-proxy" Sep 09 06:44:52.101874 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.101854 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe89f009-e9d3-404a-b39a-895d216f3c22" containerName="registry" Sep 09 06:44:52.101874 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.101859 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe89f009-e9d3-404a-b39a-895d216f3c22" containerName="registry" Sep 09 06:44:52.101874 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.101870 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7580ab0d-f488-4191-9eea-b8d56bc21ded" containerName="console" Sep 09 06:44:52.101874 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.101875 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7580ab0d-f488-4191-9eea-b8d56bc21ded" containerName="console" Sep 09 06:44:52.102110 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.101883 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73fabe44-d13f-4841-8196-f0080f9d24e7" containerName="init-config-reloader" Sep 09 06:44:52.102110 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.101889 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="73fabe44-d13f-4841-8196-f0080f9d24e7" containerName="init-config-reloader" Sep 09 06:44:52.102110 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.101896 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73fabe44-d13f-4841-8196-f0080f9d24e7" containerName="kube-rbac-proxy-web" Sep 09 06:44:52.102110 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.101901 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="73fabe44-d13f-4841-8196-f0080f9d24e7" containerName="kube-rbac-proxy-web" Sep 09 06:44:52.102110 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.101908 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73fabe44-d13f-4841-8196-f0080f9d24e7" containerName="alertmanager" Sep 09 06:44:52.102110 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.101913 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="73fabe44-d13f-4841-8196-f0080f9d24e7" containerName="alertmanager" Sep 09 06:44:52.102110 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.101923 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73fabe44-d13f-4841-8196-f0080f9d24e7" containerName="kube-rbac-proxy-metric" Sep 09 06:44:52.102110 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.101929 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="73fabe44-d13f-4841-8196-f0080f9d24e7" containerName="kube-rbac-proxy-metric" Sep 09 06:44:52.102110 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.101936 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73fabe44-d13f-4841-8196-f0080f9d24e7" containerName="config-reloader" Sep 09 06:44:52.102110 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.101942 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="73fabe44-d13f-4841-8196-f0080f9d24e7" containerName="config-reloader" Sep 09 06:44:52.102110 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.101989 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="73fabe44-d13f-4841-8196-f0080f9d24e7" containerName="kube-rbac-proxy-metric" Sep 09 06:44:52.102110 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.101997 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7580ab0d-f488-4191-9eea-b8d56bc21ded" containerName="console" Sep 09 06:44:52.102110 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.102002 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="73fabe44-d13f-4841-8196-f0080f9d24e7" containerName="prom-label-proxy" Sep 09 06:44:52.102110 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.102010 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="73fabe44-d13f-4841-8196-f0080f9d24e7" containerName="config-reloader" Sep 09 06:44:52.102110 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.102016 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe89f009-e9d3-404a-b39a-895d216f3c22" containerName="registry" Sep 09 06:44:52.102110 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.102022 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="73fabe44-d13f-4841-8196-f0080f9d24e7" containerName="alertmanager" Sep 09 06:44:52.102110 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.102027 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="73fabe44-d13f-4841-8196-f0080f9d24e7" containerName="kube-rbac-proxy-web" Sep 09 06:44:52.102110 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.102032 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="73fabe44-d13f-4841-8196-f0080f9d24e7" containerName="kube-rbac-proxy" Sep 09 06:44:52.105490 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.105475 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.112781 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.112765 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 09 06:44:52.113205 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.113189 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 09 06:44:52.113434 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.113417 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 09 06:44:52.113512 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.113500 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-8ldzq\"" Sep 09 06:44:52.113630 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.113617 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 09 06:44:52.113694 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.113655 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 09 06:44:52.113754 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.113665 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 09 06:44:52.115639 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.115619 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 09 06:44:52.117474 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.117449 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 09 06:44:52.125724 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.125703 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 09 06:44:52.139917 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.139894 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 06:44:52.213815 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.213787 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e4040508-57f2-404e-87bf-0406cd190e56-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.213815 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.213820 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e4040508-57f2-404e-87bf-0406cd190e56-web-config\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.214021 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.213840 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqfqx\" (UniqueName: \"kubernetes.io/projected/e4040508-57f2-404e-87bf-0406cd190e56-kube-api-access-gqfqx\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.214021 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.213893 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e4040508-57f2-404e-87bf-0406cd190e56-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.214021 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.213910 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4040508-57f2-404e-87bf-0406cd190e56-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.214021 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.213952 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e4040508-57f2-404e-87bf-0406cd190e56-config-out\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.214021 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.213968 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e4040508-57f2-404e-87bf-0406cd190e56-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.214021 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.214017 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e4040508-57f2-404e-87bf-0406cd190e56-config-volume\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.214337 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.214039 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e4040508-57f2-404e-87bf-0406cd190e56-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.214337 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.214079 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e4040508-57f2-404e-87bf-0406cd190e56-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.214337 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.214096 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e4040508-57f2-404e-87bf-0406cd190e56-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.214337 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.214144 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e4040508-57f2-404e-87bf-0406cd190e56-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.214337 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.214165 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e4040508-57f2-404e-87bf-0406cd190e56-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.315104 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.315002 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e4040508-57f2-404e-87bf-0406cd190e56-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.315104 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.315042 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4040508-57f2-404e-87bf-0406cd190e56-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.315104 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.315101 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e4040508-57f2-404e-87bf-0406cd190e56-config-out\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.315633 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.315127 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e4040508-57f2-404e-87bf-0406cd190e56-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.315633 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.315156 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e4040508-57f2-404e-87bf-0406cd190e56-config-volume\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.315633 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.315177 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e4040508-57f2-404e-87bf-0406cd190e56-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.315633 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.315294 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e4040508-57f2-404e-87bf-0406cd190e56-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.315633 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.315337 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e4040508-57f2-404e-87bf-0406cd190e56-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.315633 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.315382 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e4040508-57f2-404e-87bf-0406cd190e56-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.315633 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.315417 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e4040508-57f2-404e-87bf-0406cd190e56-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.315633 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.315470 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e4040508-57f2-404e-87bf-0406cd190e56-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.315633 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.315477 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e4040508-57f2-404e-87bf-0406cd190e56-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.315633 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.315501 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e4040508-57f2-404e-87bf-0406cd190e56-web-config\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.315633 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.315533 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqfqx\" (UniqueName: \"kubernetes.io/projected/e4040508-57f2-404e-87bf-0406cd190e56-kube-api-access-gqfqx\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.316268 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.315842 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e4040508-57f2-404e-87bf-0406cd190e56-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.316268 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.316037 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4040508-57f2-404e-87bf-0406cd190e56-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.318329 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.317945 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e4040508-57f2-404e-87bf-0406cd190e56-config-out\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.318329 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.318165 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e4040508-57f2-404e-87bf-0406cd190e56-config-volume\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.318329 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.318257 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e4040508-57f2-404e-87bf-0406cd190e56-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.318588 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.318564 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e4040508-57f2-404e-87bf-0406cd190e56-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.318721 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.318697 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e4040508-57f2-404e-87bf-0406cd190e56-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.318882 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.318863 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e4040508-57f2-404e-87bf-0406cd190e56-web-config\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.318951 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.318902 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e4040508-57f2-404e-87bf-0406cd190e56-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.319011 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.318981 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e4040508-57f2-404e-87bf-0406cd190e56-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.320284 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.320265 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e4040508-57f2-404e-87bf-0406cd190e56-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.323434 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.323415 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqfqx\" (UniqueName: \"kubernetes.io/projected/e4040508-57f2-404e-87bf-0406cd190e56-kube-api-access-gqfqx\") pod \"alertmanager-main-0\" (UID: \"e4040508-57f2-404e-87bf-0406cd190e56\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.423497 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.423462 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 09 06:44:52.703082 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.703031 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73fabe44-d13f-4841-8196-f0080f9d24e7" path="/var/lib/kubelet/pods/73fabe44-d13f-4841-8196-f0080f9d24e7/volumes" Sep 09 06:44:52.703760 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:52.703740 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 06:44:52.704395 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:44:52.704372 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4040508_57f2_404e_87bf_0406cd190e56.slice/crio-cd28f04582e196b49da391c6513d40b25d32e42c1ffcb2e8f42d3c9c76586552 WatchSource:0}: Error finding container cd28f04582e196b49da391c6513d40b25d32e42c1ffcb2e8f42d3c9c76586552: Status 404 returned error can't find the container with id cd28f04582e196b49da391c6513d40b25d32e42c1ffcb2e8f42d3c9c76586552 Sep 09 06:44:53.047960 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:53.047870 2559 generic.go:356] "Generic (PLEG): container finished" podID="e4040508-57f2-404e-87bf-0406cd190e56" containerID="2fa4e890a3104b21a4926488979871958b8b7afc6b26220ef20d898091cc60fa" exitCode=0 Sep 09 06:44:53.048138 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:53.047955 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e4040508-57f2-404e-87bf-0406cd190e56","Type":"ContainerDied","Data":"2fa4e890a3104b21a4926488979871958b8b7afc6b26220ef20d898091cc60fa"} Sep 09 06:44:53.048138 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:53.048021 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e4040508-57f2-404e-87bf-0406cd190e56","Type":"ContainerStarted","Data":"cd28f04582e196b49da391c6513d40b25d32e42c1ffcb2e8f42d3c9c76586552"} Sep 09 06:44:54.060671 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.060631 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e4040508-57f2-404e-87bf-0406cd190e56","Type":"ContainerStarted","Data":"3847fd0b903c96d0422d40bf670751175eecc04cab16a064234178c5d017b023"} Sep 09 06:44:54.061264 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.060718 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e4040508-57f2-404e-87bf-0406cd190e56","Type":"ContainerStarted","Data":"5a517ce1a4b491feecd5a8a2175a711040f0b0354b0d1a954f655c63563d9f02"} Sep 09 06:44:54.061264 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.060750 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e4040508-57f2-404e-87bf-0406cd190e56","Type":"ContainerStarted","Data":"f0edbdb025a1b740d73c188f6c83e1e54e82fccfa94ae4bc8c0a6eb2903e4fb1"} Sep 09 06:44:54.061264 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.060785 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e4040508-57f2-404e-87bf-0406cd190e56","Type":"ContainerStarted","Data":"aa941713f5c953918a1f9263c3356382d5febf7de0bc57cc2774d2f99707cece"} Sep 09 06:44:54.061264 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.060821 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e4040508-57f2-404e-87bf-0406cd190e56","Type":"ContainerStarted","Data":"58f00946cf0aa29e0df88165931446ed85fbc0fdde4fe2c5def1a847b7f016bd"} Sep 09 06:44:54.061264 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.060860 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e4040508-57f2-404e-87bf-0406cd190e56","Type":"ContainerStarted","Data":"671d6427eefd64f3b17f857c5b3db078802f1fc1d57c91d169df558bc0876c6e"} Sep 09 06:44:54.077046 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.077006 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-546fff794d-2vkft"] Sep 09 06:44:54.079861 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.079837 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-546fff794d-2vkft" Sep 09 06:44:54.082190 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.082164 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 09 06:44:54.082309 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.082206 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 09 06:44:54.082494 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.082477 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-pfvhk\"" Sep 09 06:44:54.082560 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.082494 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 09 06:44:54.082560 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.082513 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 09 06:44:54.082641 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.082623 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 09 06:44:54.088396 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.088379 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 09 06:44:54.094580 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.094560 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-546fff794d-2vkft"] Sep 09 06:44:54.096933 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.096892 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.08005856 podStartE2EDuration="2.096877845s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.819285ms" isStatefulPod=false podCreationTimestamp="2026-09-09 06:44:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:44:54.095393755 +0000 UTC m=+391.991811180" watchObservedRunningTime="2026-09-09 06:44:54.096877845 +0000 UTC m=+391.993295267" Sep 09 06:44:54.232816 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.231714 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tw86v\" (UniqueName: \"kubernetes.io/projected/8987da44-4b1d-4a54-8081-1a70de01ed92-kube-api-access-tw86v\") pod \"telemeter-client-546fff794d-2vkft\" (UID: \"8987da44-4b1d-4a54-8081-1a70de01ed92\") " pod="openshift-monitoring/telemeter-client-546fff794d-2vkft" Sep 09 06:44:54.232816 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.231767 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8987da44-4b1d-4a54-8081-1a70de01ed92-telemeter-trusted-ca-bundle\") pod \"telemeter-client-546fff794d-2vkft\" (UID: \"8987da44-4b1d-4a54-8081-1a70de01ed92\") " pod="openshift-monitoring/telemeter-client-546fff794d-2vkft" Sep 09 06:44:54.232816 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.231867 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8987da44-4b1d-4a54-8081-1a70de01ed92-secret-telemeter-client\") pod \"telemeter-client-546fff794d-2vkft\" (UID: \"8987da44-4b1d-4a54-8081-1a70de01ed92\") " pod="openshift-monitoring/telemeter-client-546fff794d-2vkft" Sep 09 06:44:54.232816 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.231932 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8987da44-4b1d-4a54-8081-1a70de01ed92-serving-certs-ca-bundle\") pod \"telemeter-client-546fff794d-2vkft\" (UID: \"8987da44-4b1d-4a54-8081-1a70de01ed92\") " pod="openshift-monitoring/telemeter-client-546fff794d-2vkft" Sep 09 06:44:54.232816 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.231979 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8987da44-4b1d-4a54-8081-1a70de01ed92-federate-client-tls\") pod \"telemeter-client-546fff794d-2vkft\" (UID: \"8987da44-4b1d-4a54-8081-1a70de01ed92\") " pod="openshift-monitoring/telemeter-client-546fff794d-2vkft" Sep 09 06:44:54.232816 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.232030 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8987da44-4b1d-4a54-8081-1a70de01ed92-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-546fff794d-2vkft\" (UID: \"8987da44-4b1d-4a54-8081-1a70de01ed92\") " pod="openshift-monitoring/telemeter-client-546fff794d-2vkft" Sep 09 06:44:54.232816 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.232108 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8987da44-4b1d-4a54-8081-1a70de01ed92-telemeter-client-tls\") pod \"telemeter-client-546fff794d-2vkft\" (UID: \"8987da44-4b1d-4a54-8081-1a70de01ed92\") " pod="openshift-monitoring/telemeter-client-546fff794d-2vkft" Sep 09 06:44:54.232816 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.232164 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8987da44-4b1d-4a54-8081-1a70de01ed92-metrics-client-ca\") pod \"telemeter-client-546fff794d-2vkft\" (UID: \"8987da44-4b1d-4a54-8081-1a70de01ed92\") " pod="openshift-monitoring/telemeter-client-546fff794d-2vkft" Sep 09 06:44:54.333483 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.333394 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw86v\" (UniqueName: \"kubernetes.io/projected/8987da44-4b1d-4a54-8081-1a70de01ed92-kube-api-access-tw86v\") pod \"telemeter-client-546fff794d-2vkft\" (UID: \"8987da44-4b1d-4a54-8081-1a70de01ed92\") " pod="openshift-monitoring/telemeter-client-546fff794d-2vkft" Sep 09 06:44:54.333483 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.333433 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8987da44-4b1d-4a54-8081-1a70de01ed92-telemeter-trusted-ca-bundle\") pod \"telemeter-client-546fff794d-2vkft\" (UID: \"8987da44-4b1d-4a54-8081-1a70de01ed92\") " pod="openshift-monitoring/telemeter-client-546fff794d-2vkft" Sep 09 06:44:54.333672 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.333568 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8987da44-4b1d-4a54-8081-1a70de01ed92-secret-telemeter-client\") pod \"telemeter-client-546fff794d-2vkft\" (UID: \"8987da44-4b1d-4a54-8081-1a70de01ed92\") " pod="openshift-monitoring/telemeter-client-546fff794d-2vkft" Sep 09 06:44:54.333672 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.333608 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8987da44-4b1d-4a54-8081-1a70de01ed92-serving-certs-ca-bundle\") pod \"telemeter-client-546fff794d-2vkft\" (UID: \"8987da44-4b1d-4a54-8081-1a70de01ed92\") " pod="openshift-monitoring/telemeter-client-546fff794d-2vkft" Sep 09 06:44:54.333672 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.333629 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8987da44-4b1d-4a54-8081-1a70de01ed92-federate-client-tls\") pod \"telemeter-client-546fff794d-2vkft\" (UID: \"8987da44-4b1d-4a54-8081-1a70de01ed92\") " pod="openshift-monitoring/telemeter-client-546fff794d-2vkft" Sep 09 06:44:54.333826 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.333765 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8987da44-4b1d-4a54-8081-1a70de01ed92-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-546fff794d-2vkft\" (UID: \"8987da44-4b1d-4a54-8081-1a70de01ed92\") " pod="openshift-monitoring/telemeter-client-546fff794d-2vkft" Sep 09 06:44:54.333826 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.333809 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8987da44-4b1d-4a54-8081-1a70de01ed92-telemeter-client-tls\") pod \"telemeter-client-546fff794d-2vkft\" (UID: \"8987da44-4b1d-4a54-8081-1a70de01ed92\") " pod="openshift-monitoring/telemeter-client-546fff794d-2vkft" Sep 09 06:44:54.333926 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.333849 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8987da44-4b1d-4a54-8081-1a70de01ed92-metrics-client-ca\") pod \"telemeter-client-546fff794d-2vkft\" (UID: \"8987da44-4b1d-4a54-8081-1a70de01ed92\") " pod="openshift-monitoring/telemeter-client-546fff794d-2vkft" Sep 09 06:44:54.334403 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.334372 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8987da44-4b1d-4a54-8081-1a70de01ed92-serving-certs-ca-bundle\") pod \"telemeter-client-546fff794d-2vkft\" (UID: \"8987da44-4b1d-4a54-8081-1a70de01ed92\") " pod="openshift-monitoring/telemeter-client-546fff794d-2vkft" Sep 09 06:44:54.334531 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.334464 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8987da44-4b1d-4a54-8081-1a70de01ed92-telemeter-trusted-ca-bundle\") pod \"telemeter-client-546fff794d-2vkft\" (UID: \"8987da44-4b1d-4a54-8081-1a70de01ed92\") " pod="openshift-monitoring/telemeter-client-546fff794d-2vkft" Sep 09 06:44:54.334531 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.334490 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8987da44-4b1d-4a54-8081-1a70de01ed92-metrics-client-ca\") pod \"telemeter-client-546fff794d-2vkft\" (UID: \"8987da44-4b1d-4a54-8081-1a70de01ed92\") " pod="openshift-monitoring/telemeter-client-546fff794d-2vkft" Sep 09 06:44:54.336255 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.336229 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8987da44-4b1d-4a54-8081-1a70de01ed92-secret-telemeter-client\") pod \"telemeter-client-546fff794d-2vkft\" (UID: \"8987da44-4b1d-4a54-8081-1a70de01ed92\") " pod="openshift-monitoring/telemeter-client-546fff794d-2vkft" Sep 09 06:44:54.336361 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.336341 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8987da44-4b1d-4a54-8081-1a70de01ed92-federate-client-tls\") pod \"telemeter-client-546fff794d-2vkft\" (UID: \"8987da44-4b1d-4a54-8081-1a70de01ed92\") " pod="openshift-monitoring/telemeter-client-546fff794d-2vkft" Sep 09 06:44:54.336479 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.336459 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8987da44-4b1d-4a54-8081-1a70de01ed92-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-546fff794d-2vkft\" (UID: \"8987da44-4b1d-4a54-8081-1a70de01ed92\") " pod="openshift-monitoring/telemeter-client-546fff794d-2vkft" Sep 09 06:44:54.336533 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.336515 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8987da44-4b1d-4a54-8081-1a70de01ed92-telemeter-client-tls\") pod \"telemeter-client-546fff794d-2vkft\" (UID: \"8987da44-4b1d-4a54-8081-1a70de01ed92\") " pod="openshift-monitoring/telemeter-client-546fff794d-2vkft" Sep 09 06:44:54.340381 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.340355 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tw86v\" (UniqueName: \"kubernetes.io/projected/8987da44-4b1d-4a54-8081-1a70de01ed92-kube-api-access-tw86v\") pod \"telemeter-client-546fff794d-2vkft\" (UID: \"8987da44-4b1d-4a54-8081-1a70de01ed92\") " pod="openshift-monitoring/telemeter-client-546fff794d-2vkft" Sep 09 06:44:54.391181 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.391142 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-546fff794d-2vkft" Sep 09 06:44:54.681084 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:54.680884 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-546fff794d-2vkft"] Sep 09 06:44:54.683068 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:44:54.683030 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8987da44_4b1d_4a54_8081_1a70de01ed92.slice/crio-61a69ba932d681ddfce7dad0dbda55c163fb5056044983e81f28ab3af058afcd WatchSource:0}: Error finding container 61a69ba932d681ddfce7dad0dbda55c163fb5056044983e81f28ab3af058afcd: Status 404 returned error can't find the container with id 61a69ba932d681ddfce7dad0dbda55c163fb5056044983e81f28ab3af058afcd Sep 09 06:44:55.065686 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:55.065607 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-546fff794d-2vkft" event={"ID":"8987da44-4b1d-4a54-8081-1a70de01ed92","Type":"ContainerStarted","Data":"61a69ba932d681ddfce7dad0dbda55c163fb5056044983e81f28ab3af058afcd"} Sep 09 06:44:57.078000 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:57.077908 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-546fff794d-2vkft" event={"ID":"8987da44-4b1d-4a54-8081-1a70de01ed92","Type":"ContainerStarted","Data":"449c82d85ee639d70d5a2443fc656186419d42a8114d7970957e327443b78a45"} Sep 09 06:44:57.078000 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:57.077969 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-546fff794d-2vkft" event={"ID":"8987da44-4b1d-4a54-8081-1a70de01ed92","Type":"ContainerStarted","Data":"9ccea5c8ce73c428fedbbdf8f10200df7d1050c0263d4e552fe86cbb386fa8cf"} Sep 09 06:44:57.078000 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:57.077997 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-546fff794d-2vkft" event={"ID":"8987da44-4b1d-4a54-8081-1a70de01ed92","Type":"ContainerStarted","Data":"d6f7ad18e777441fdd008c710c397d401e66a4ce3774992dea7d348d8c27c25f"} Sep 09 06:44:57.099828 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:57.099785 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-546fff794d-2vkft" podStartSLOduration=1.063318913 podStartE2EDuration="3.099767334s" totalImagesPullingTime="2.036448421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:44:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:44:57.098776543 +0000 UTC m=+394.995193965" watchObservedRunningTime="2026-09-09 06:44:57.099767334 +0000 UTC m=+394.996184760" Sep 09 06:44:57.202235 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:57.202199 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d976b9bf8-nxpnt"] Sep 09 06:44:57.205087 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:57.205047 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d976b9bf8-nxpnt" Sep 09 06:44:57.207684 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:57.207658 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 09 06:44:57.207988 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:57.207948 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 09 06:44:57.208546 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:57.208523 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 09 06:44:57.209233 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:57.209213 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 09 06:44:57.211117 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:57.211085 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 09 06:44:57.211117 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:57.211113 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-58spp\"" Sep 09 06:44:57.216141 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:57.216115 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 09 06:44:57.223546 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:57.223520 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d976b9bf8-nxpnt"] Sep 09 06:44:57.361234 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:57.361151 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0c2234cd-5566-460a-ad8e-7cbbf25f805e-console-oauth-config\") pod \"console-6d976b9bf8-nxpnt\" (UID: \"0c2234cd-5566-460a-ad8e-7cbbf25f805e\") " pod="openshift-console/console-6d976b9bf8-nxpnt" Sep 09 06:44:57.361234 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:57.361186 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0c2234cd-5566-460a-ad8e-7cbbf25f805e-console-config\") pod \"console-6d976b9bf8-nxpnt\" (UID: \"0c2234cd-5566-460a-ad8e-7cbbf25f805e\") " pod="openshift-console/console-6d976b9bf8-nxpnt" Sep 09 06:44:57.361234 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:57.361208 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c2234cd-5566-460a-ad8e-7cbbf25f805e-trusted-ca-bundle\") pod \"console-6d976b9bf8-nxpnt\" (UID: \"0c2234cd-5566-460a-ad8e-7cbbf25f805e\") " pod="openshift-console/console-6d976b9bf8-nxpnt" Sep 09 06:44:57.361494 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:57.361260 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vw222\" (UniqueName: \"kubernetes.io/projected/0c2234cd-5566-460a-ad8e-7cbbf25f805e-kube-api-access-vw222\") pod \"console-6d976b9bf8-nxpnt\" (UID: \"0c2234cd-5566-460a-ad8e-7cbbf25f805e\") " pod="openshift-console/console-6d976b9bf8-nxpnt" Sep 09 06:44:57.361494 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:57.361305 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0c2234cd-5566-460a-ad8e-7cbbf25f805e-oauth-serving-cert\") pod \"console-6d976b9bf8-nxpnt\" (UID: \"0c2234cd-5566-460a-ad8e-7cbbf25f805e\") " pod="openshift-console/console-6d976b9bf8-nxpnt" Sep 09 06:44:57.361494 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:57.361352 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0c2234cd-5566-460a-ad8e-7cbbf25f805e-console-serving-cert\") pod \"console-6d976b9bf8-nxpnt\" (UID: \"0c2234cd-5566-460a-ad8e-7cbbf25f805e\") " pod="openshift-console/console-6d976b9bf8-nxpnt" Sep 09 06:44:57.361494 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:57.361377 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0c2234cd-5566-460a-ad8e-7cbbf25f805e-service-ca\") pod \"console-6d976b9bf8-nxpnt\" (UID: \"0c2234cd-5566-460a-ad8e-7cbbf25f805e\") " pod="openshift-console/console-6d976b9bf8-nxpnt" Sep 09 06:44:57.462473 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:57.462442 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c2234cd-5566-460a-ad8e-7cbbf25f805e-trusted-ca-bundle\") pod \"console-6d976b9bf8-nxpnt\" (UID: \"0c2234cd-5566-460a-ad8e-7cbbf25f805e\") " pod="openshift-console/console-6d976b9bf8-nxpnt" Sep 09 06:44:57.462643 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:57.462490 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vw222\" (UniqueName: \"kubernetes.io/projected/0c2234cd-5566-460a-ad8e-7cbbf25f805e-kube-api-access-vw222\") pod \"console-6d976b9bf8-nxpnt\" (UID: \"0c2234cd-5566-460a-ad8e-7cbbf25f805e\") " pod="openshift-console/console-6d976b9bf8-nxpnt" Sep 09 06:44:57.462643 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:57.462517 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0c2234cd-5566-460a-ad8e-7cbbf25f805e-oauth-serving-cert\") pod \"console-6d976b9bf8-nxpnt\" (UID: \"0c2234cd-5566-460a-ad8e-7cbbf25f805e\") " pod="openshift-console/console-6d976b9bf8-nxpnt" Sep 09 06:44:57.462643 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:57.462547 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0c2234cd-5566-460a-ad8e-7cbbf25f805e-console-serving-cert\") pod \"console-6d976b9bf8-nxpnt\" (UID: \"0c2234cd-5566-460a-ad8e-7cbbf25f805e\") " pod="openshift-console/console-6d976b9bf8-nxpnt" Sep 09 06:44:57.462643 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:57.462572 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0c2234cd-5566-460a-ad8e-7cbbf25f805e-service-ca\") pod \"console-6d976b9bf8-nxpnt\" (UID: \"0c2234cd-5566-460a-ad8e-7cbbf25f805e\") " pod="openshift-console/console-6d976b9bf8-nxpnt" Sep 09 06:44:57.462643 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:57.462641 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0c2234cd-5566-460a-ad8e-7cbbf25f805e-console-oauth-config\") pod \"console-6d976b9bf8-nxpnt\" (UID: \"0c2234cd-5566-460a-ad8e-7cbbf25f805e\") " pod="openshift-console/console-6d976b9bf8-nxpnt" Sep 09 06:44:57.462871 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:57.462665 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0c2234cd-5566-460a-ad8e-7cbbf25f805e-console-config\") pod \"console-6d976b9bf8-nxpnt\" (UID: \"0c2234cd-5566-460a-ad8e-7cbbf25f805e\") " pod="openshift-console/console-6d976b9bf8-nxpnt" Sep 09 06:44:57.463336 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:57.463311 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0c2234cd-5566-460a-ad8e-7cbbf25f805e-oauth-serving-cert\") pod \"console-6d976b9bf8-nxpnt\" (UID: \"0c2234cd-5566-460a-ad8e-7cbbf25f805e\") " pod="openshift-console/console-6d976b9bf8-nxpnt" Sep 09 06:44:57.463450 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:57.463348 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0c2234cd-5566-460a-ad8e-7cbbf25f805e-service-ca\") pod \"console-6d976b9bf8-nxpnt\" (UID: \"0c2234cd-5566-460a-ad8e-7cbbf25f805e\") " pod="openshift-console/console-6d976b9bf8-nxpnt" Sep 09 06:44:57.463450 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:57.463411 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c2234cd-5566-460a-ad8e-7cbbf25f805e-trusted-ca-bundle\") pod \"console-6d976b9bf8-nxpnt\" (UID: \"0c2234cd-5566-460a-ad8e-7cbbf25f805e\") " pod="openshift-console/console-6d976b9bf8-nxpnt" Sep 09 06:44:57.463450 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:57.463420 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0c2234cd-5566-460a-ad8e-7cbbf25f805e-console-config\") pod \"console-6d976b9bf8-nxpnt\" (UID: \"0c2234cd-5566-460a-ad8e-7cbbf25f805e\") " pod="openshift-console/console-6d976b9bf8-nxpnt" Sep 09 06:44:57.465164 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:57.465139 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0c2234cd-5566-460a-ad8e-7cbbf25f805e-console-serving-cert\") pod \"console-6d976b9bf8-nxpnt\" (UID: \"0c2234cd-5566-460a-ad8e-7cbbf25f805e\") " pod="openshift-console/console-6d976b9bf8-nxpnt" Sep 09 06:44:57.465265 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:57.465228 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0c2234cd-5566-460a-ad8e-7cbbf25f805e-console-oauth-config\") pod \"console-6d976b9bf8-nxpnt\" (UID: \"0c2234cd-5566-460a-ad8e-7cbbf25f805e\") " pod="openshift-console/console-6d976b9bf8-nxpnt" Sep 09 06:44:57.470046 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:57.470026 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vw222\" (UniqueName: \"kubernetes.io/projected/0c2234cd-5566-460a-ad8e-7cbbf25f805e-kube-api-access-vw222\") pod \"console-6d976b9bf8-nxpnt\" (UID: \"0c2234cd-5566-460a-ad8e-7cbbf25f805e\") " pod="openshift-console/console-6d976b9bf8-nxpnt" Sep 09 06:44:57.516972 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:57.516934 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d976b9bf8-nxpnt" Sep 09 06:44:57.790960 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:57.790925 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d976b9bf8-nxpnt"] Sep 09 06:44:57.794841 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:44:57.794813 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c2234cd_5566_460a_ad8e_7cbbf25f805e.slice/crio-feb194b2cc55092522711e3cd8339da75099ffca0dbf5bbdba09f9f48df61434 WatchSource:0}: Error finding container feb194b2cc55092522711e3cd8339da75099ffca0dbf5bbdba09f9f48df61434: Status 404 returned error can't find the container with id feb194b2cc55092522711e3cd8339da75099ffca0dbf5bbdba09f9f48df61434 Sep 09 06:44:58.085121 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:58.084992 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d976b9bf8-nxpnt" event={"ID":"0c2234cd-5566-460a-ad8e-7cbbf25f805e","Type":"ContainerStarted","Data":"ce6aa0e7046bb7baca0cc61443839a593fce5b4e801261409b0bf97ff160feb3"} Sep 09 06:44:58.085121 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:58.085075 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d976b9bf8-nxpnt" event={"ID":"0c2234cd-5566-460a-ad8e-7cbbf25f805e","Type":"ContainerStarted","Data":"feb194b2cc55092522711e3cd8339da75099ffca0dbf5bbdba09f9f48df61434"} Sep 09 06:44:58.109673 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:44:58.109614 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6d976b9bf8-nxpnt" podStartSLOduration=1.109597824 podStartE2EDuration="1.109597824s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:44:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:44:58.107809249 +0000 UTC m=+396.004226672" watchObservedRunningTime="2026-09-09 06:44:58.109597824 +0000 UTC m=+396.006015322" Sep 09 06:45:07.517119 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:45:07.517081 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6d976b9bf8-nxpnt" Sep 09 06:45:07.517508 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:45:07.517151 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d976b9bf8-nxpnt" Sep 09 06:45:07.521709 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:45:07.521685 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6d976b9bf8-nxpnt" Sep 09 06:45:08.139961 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:45:08.139931 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6d976b9bf8-nxpnt" Sep 09 06:46:07.061678 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:07.061638 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b597d587b-g2vg8"] Sep 09 06:46:07.065798 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:07.065771 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b597d587b-g2vg8" Sep 09 06:46:07.077338 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:07.077311 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b597d587b-g2vg8"] Sep 09 06:46:07.131951 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:07.131914 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw54q\" (UniqueName: \"kubernetes.io/projected/e9a7cc46-4f97-4021-a031-b990097ff56e-kube-api-access-xw54q\") pod \"console-5b597d587b-g2vg8\" (UID: \"e9a7cc46-4f97-4021-a031-b990097ff56e\") " pod="openshift-console/console-5b597d587b-g2vg8" Sep 09 06:46:07.131951 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:07.131952 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e9a7cc46-4f97-4021-a031-b990097ff56e-console-oauth-config\") pod \"console-5b597d587b-g2vg8\" (UID: \"e9a7cc46-4f97-4021-a031-b990097ff56e\") " pod="openshift-console/console-5b597d587b-g2vg8" Sep 09 06:46:07.132174 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:07.131972 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e9a7cc46-4f97-4021-a031-b990097ff56e-oauth-serving-cert\") pod \"console-5b597d587b-g2vg8\" (UID: \"e9a7cc46-4f97-4021-a031-b990097ff56e\") " pod="openshift-console/console-5b597d587b-g2vg8" Sep 09 06:46:07.132174 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:07.131996 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e9a7cc46-4f97-4021-a031-b990097ff56e-console-config\") pod \"console-5b597d587b-g2vg8\" (UID: \"e9a7cc46-4f97-4021-a031-b990097ff56e\") " pod="openshift-console/console-5b597d587b-g2vg8" Sep 09 06:46:07.132174 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:07.132040 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9a7cc46-4f97-4021-a031-b990097ff56e-trusted-ca-bundle\") pod \"console-5b597d587b-g2vg8\" (UID: \"e9a7cc46-4f97-4021-a031-b990097ff56e\") " pod="openshift-console/console-5b597d587b-g2vg8" Sep 09 06:46:07.132174 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:07.132090 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e9a7cc46-4f97-4021-a031-b990097ff56e-service-ca\") pod \"console-5b597d587b-g2vg8\" (UID: \"e9a7cc46-4f97-4021-a031-b990097ff56e\") " pod="openshift-console/console-5b597d587b-g2vg8" Sep 09 06:46:07.132174 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:07.132105 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e9a7cc46-4f97-4021-a031-b990097ff56e-console-serving-cert\") pod \"console-5b597d587b-g2vg8\" (UID: \"e9a7cc46-4f97-4021-a031-b990097ff56e\") " pod="openshift-console/console-5b597d587b-g2vg8" Sep 09 06:46:07.232688 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:07.232654 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e9a7cc46-4f97-4021-a031-b990097ff56e-service-ca\") pod \"console-5b597d587b-g2vg8\" (UID: \"e9a7cc46-4f97-4021-a031-b990097ff56e\") " pod="openshift-console/console-5b597d587b-g2vg8" Sep 09 06:46:07.232688 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:07.232686 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e9a7cc46-4f97-4021-a031-b990097ff56e-console-serving-cert\") pod \"console-5b597d587b-g2vg8\" (UID: \"e9a7cc46-4f97-4021-a031-b990097ff56e\") " pod="openshift-console/console-5b597d587b-g2vg8" Sep 09 06:46:07.232927 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:07.232825 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw54q\" (UniqueName: \"kubernetes.io/projected/e9a7cc46-4f97-4021-a031-b990097ff56e-kube-api-access-xw54q\") pod \"console-5b597d587b-g2vg8\" (UID: \"e9a7cc46-4f97-4021-a031-b990097ff56e\") " pod="openshift-console/console-5b597d587b-g2vg8" Sep 09 06:46:07.232927 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:07.232869 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e9a7cc46-4f97-4021-a031-b990097ff56e-console-oauth-config\") pod \"console-5b597d587b-g2vg8\" (UID: \"e9a7cc46-4f97-4021-a031-b990097ff56e\") " pod="openshift-console/console-5b597d587b-g2vg8" Sep 09 06:46:07.232927 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:07.232901 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e9a7cc46-4f97-4021-a031-b990097ff56e-oauth-serving-cert\") pod \"console-5b597d587b-g2vg8\" (UID: \"e9a7cc46-4f97-4021-a031-b990097ff56e\") " pod="openshift-console/console-5b597d587b-g2vg8" Sep 09 06:46:07.233104 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:07.232932 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e9a7cc46-4f97-4021-a031-b990097ff56e-console-config\") pod \"console-5b597d587b-g2vg8\" (UID: \"e9a7cc46-4f97-4021-a031-b990097ff56e\") " pod="openshift-console/console-5b597d587b-g2vg8" Sep 09 06:46:07.233168 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:07.233135 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9a7cc46-4f97-4021-a031-b990097ff56e-trusted-ca-bundle\") pod \"console-5b597d587b-g2vg8\" (UID: \"e9a7cc46-4f97-4021-a031-b990097ff56e\") " pod="openshift-console/console-5b597d587b-g2vg8" Sep 09 06:46:07.233640 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:07.233613 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e9a7cc46-4f97-4021-a031-b990097ff56e-service-ca\") pod \"console-5b597d587b-g2vg8\" (UID: \"e9a7cc46-4f97-4021-a031-b990097ff56e\") " pod="openshift-console/console-5b597d587b-g2vg8" Sep 09 06:46:07.233756 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:07.233671 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e9a7cc46-4f97-4021-a031-b990097ff56e-console-config\") pod \"console-5b597d587b-g2vg8\" (UID: \"e9a7cc46-4f97-4021-a031-b990097ff56e\") " pod="openshift-console/console-5b597d587b-g2vg8" Sep 09 06:46:07.233756 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:07.233725 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e9a7cc46-4f97-4021-a031-b990097ff56e-oauth-serving-cert\") pod \"console-5b597d587b-g2vg8\" (UID: \"e9a7cc46-4f97-4021-a031-b990097ff56e\") " pod="openshift-console/console-5b597d587b-g2vg8" Sep 09 06:46:07.233839 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:07.233810 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9a7cc46-4f97-4021-a031-b990097ff56e-trusted-ca-bundle\") pod \"console-5b597d587b-g2vg8\" (UID: \"e9a7cc46-4f97-4021-a031-b990097ff56e\") " pod="openshift-console/console-5b597d587b-g2vg8" Sep 09 06:46:07.235313 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:07.235282 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e9a7cc46-4f97-4021-a031-b990097ff56e-console-oauth-config\") pod \"console-5b597d587b-g2vg8\" (UID: \"e9a7cc46-4f97-4021-a031-b990097ff56e\") " pod="openshift-console/console-5b597d587b-g2vg8" Sep 09 06:46:07.235424 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:07.235401 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e9a7cc46-4f97-4021-a031-b990097ff56e-console-serving-cert\") pod \"console-5b597d587b-g2vg8\" (UID: \"e9a7cc46-4f97-4021-a031-b990097ff56e\") " pod="openshift-console/console-5b597d587b-g2vg8" Sep 09 06:46:07.240020 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:07.240003 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw54q\" (UniqueName: \"kubernetes.io/projected/e9a7cc46-4f97-4021-a031-b990097ff56e-kube-api-access-xw54q\") pod \"console-5b597d587b-g2vg8\" (UID: \"e9a7cc46-4f97-4021-a031-b990097ff56e\") " pod="openshift-console/console-5b597d587b-g2vg8" Sep 09 06:46:07.376117 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:07.376002 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b597d587b-g2vg8" Sep 09 06:46:07.651026 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:07.651002 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b597d587b-g2vg8"] Sep 09 06:46:07.653562 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:46:07.653527 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9a7cc46_4f97_4021_a031_b990097ff56e.slice/crio-4d74b4da827a1dd0e687061d1b758d9a9d6c47dbeb275f0670468f96f8387ac1 WatchSource:0}: Error finding container 4d74b4da827a1dd0e687061d1b758d9a9d6c47dbeb275f0670468f96f8387ac1: Status 404 returned error can't find the container with id 4d74b4da827a1dd0e687061d1b758d9a9d6c47dbeb275f0670468f96f8387ac1 Sep 09 06:46:08.379539 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:08.379492 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b597d587b-g2vg8" event={"ID":"e9a7cc46-4f97-4021-a031-b990097ff56e","Type":"ContainerStarted","Data":"cb58326814cacb513ff234af577b2acde49054164275f79425c68a9a3938baec"} Sep 09 06:46:08.379958 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:08.379595 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b597d587b-g2vg8" event={"ID":"e9a7cc46-4f97-4021-a031-b990097ff56e","Type":"ContainerStarted","Data":"4d74b4da827a1dd0e687061d1b758d9a9d6c47dbeb275f0670468f96f8387ac1"} Sep 09 06:46:08.397393 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:08.397341 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5b597d587b-g2vg8" podStartSLOduration=1.397322544 podStartE2EDuration="1.397322544s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:46:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:46:08.395369866 +0000 UTC m=+466.291787287" watchObservedRunningTime="2026-09-09 06:46:08.397322544 +0000 UTC m=+466.293739965" Sep 09 06:46:17.376535 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:17.376500 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5b597d587b-g2vg8" Sep 09 06:46:17.377128 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:17.376562 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b597d587b-g2vg8" Sep 09 06:46:17.381346 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:17.381323 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5b597d587b-g2vg8" Sep 09 06:46:17.421585 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:17.421563 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5b597d587b-g2vg8" Sep 09 06:46:17.466611 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:17.466583 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d976b9bf8-nxpnt"] Sep 09 06:46:42.487791 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:42.487747 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6d976b9bf8-nxpnt" podUID="0c2234cd-5566-460a-ad8e-7cbbf25f805e" containerName="console" containerID="cri-o://ce6aa0e7046bb7baca0cc61443839a593fce5b4e801261409b0bf97ff160feb3" gracePeriod=15 Sep 09 06:46:42.725563 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:42.725540 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d976b9bf8-nxpnt" Sep 09 06:46:42.733108 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:42.733082 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0c2234cd-5566-460a-ad8e-7cbbf25f805e-console-config\") pod \"0c2234cd-5566-460a-ad8e-7cbbf25f805e\" (UID: \"0c2234cd-5566-460a-ad8e-7cbbf25f805e\") " Sep 09 06:46:42.733212 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:42.733122 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c2234cd-5566-460a-ad8e-7cbbf25f805e-trusted-ca-bundle\") pod \"0c2234cd-5566-460a-ad8e-7cbbf25f805e\" (UID: \"0c2234cd-5566-460a-ad8e-7cbbf25f805e\") " Sep 09 06:46:42.733212 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:42.733160 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0c2234cd-5566-460a-ad8e-7cbbf25f805e-console-serving-cert\") pod \"0c2234cd-5566-460a-ad8e-7cbbf25f805e\" (UID: \"0c2234cd-5566-460a-ad8e-7cbbf25f805e\") " Sep 09 06:46:42.733212 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:42.733195 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vw222\" (UniqueName: \"kubernetes.io/projected/0c2234cd-5566-460a-ad8e-7cbbf25f805e-kube-api-access-vw222\") pod \"0c2234cd-5566-460a-ad8e-7cbbf25f805e\" (UID: \"0c2234cd-5566-460a-ad8e-7cbbf25f805e\") " Sep 09 06:46:42.733359 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:42.733234 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0c2234cd-5566-460a-ad8e-7cbbf25f805e-service-ca\") pod \"0c2234cd-5566-460a-ad8e-7cbbf25f805e\" (UID: \"0c2234cd-5566-460a-ad8e-7cbbf25f805e\") " Sep 09 06:46:42.733359 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:42.733268 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0c2234cd-5566-460a-ad8e-7cbbf25f805e-console-oauth-config\") pod \"0c2234cd-5566-460a-ad8e-7cbbf25f805e\" (UID: \"0c2234cd-5566-460a-ad8e-7cbbf25f805e\") " Sep 09 06:46:42.733359 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:42.733297 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0c2234cd-5566-460a-ad8e-7cbbf25f805e-oauth-serving-cert\") pod \"0c2234cd-5566-460a-ad8e-7cbbf25f805e\" (UID: \"0c2234cd-5566-460a-ad8e-7cbbf25f805e\") " Sep 09 06:46:42.733509 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:42.733458 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c2234cd-5566-460a-ad8e-7cbbf25f805e-console-config" (OuterVolumeSpecName: "console-config") pod "0c2234cd-5566-460a-ad8e-7cbbf25f805e" (UID: "0c2234cd-5566-460a-ad8e-7cbbf25f805e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 06:46:42.733674 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:42.733657 2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0c2234cd-5566-460a-ad8e-7cbbf25f805e-console-config\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:46:42.733747 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:42.733705 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c2234cd-5566-460a-ad8e-7cbbf25f805e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0c2234cd-5566-460a-ad8e-7cbbf25f805e" (UID: "0c2234cd-5566-460a-ad8e-7cbbf25f805e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 06:46:42.733747 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:42.733707 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c2234cd-5566-460a-ad8e-7cbbf25f805e-service-ca" (OuterVolumeSpecName: "service-ca") pod "0c2234cd-5566-460a-ad8e-7cbbf25f805e" (UID: "0c2234cd-5566-460a-ad8e-7cbbf25f805e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 06:46:42.733860 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:42.733838 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c2234cd-5566-460a-ad8e-7cbbf25f805e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0c2234cd-5566-460a-ad8e-7cbbf25f805e" (UID: "0c2234cd-5566-460a-ad8e-7cbbf25f805e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 06:46:42.735268 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:42.735243 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c2234cd-5566-460a-ad8e-7cbbf25f805e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0c2234cd-5566-460a-ad8e-7cbbf25f805e" (UID: "0c2234cd-5566-460a-ad8e-7cbbf25f805e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 06:46:42.735364 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:42.735325 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c2234cd-5566-460a-ad8e-7cbbf25f805e-kube-api-access-vw222" (OuterVolumeSpecName: "kube-api-access-vw222") pod "0c2234cd-5566-460a-ad8e-7cbbf25f805e" (UID: "0c2234cd-5566-460a-ad8e-7cbbf25f805e"). InnerVolumeSpecName "kube-api-access-vw222". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 06:46:42.735412 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:42.735380 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c2234cd-5566-460a-ad8e-7cbbf25f805e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0c2234cd-5566-460a-ad8e-7cbbf25f805e" (UID: "0c2234cd-5566-460a-ad8e-7cbbf25f805e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 06:46:42.834024 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:42.833991 2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c2234cd-5566-460a-ad8e-7cbbf25f805e-trusted-ca-bundle\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:46:42.834024 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:42.834019 2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0c2234cd-5566-460a-ad8e-7cbbf25f805e-console-serving-cert\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:46:42.834210 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:42.834032 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vw222\" (UniqueName: \"kubernetes.io/projected/0c2234cd-5566-460a-ad8e-7cbbf25f805e-kube-api-access-vw222\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:46:42.834210 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:42.834047 2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0c2234cd-5566-460a-ad8e-7cbbf25f805e-service-ca\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:46:42.834210 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:42.834075 2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0c2234cd-5566-460a-ad8e-7cbbf25f805e-console-oauth-config\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:46:42.834210 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:42.834086 2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0c2234cd-5566-460a-ad8e-7cbbf25f805e-oauth-serving-cert\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:46:43.528747 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:43.528712 2559 generic.go:356] "Generic (PLEG): container finished" podID="0c2234cd-5566-460a-ad8e-7cbbf25f805e" containerID="ce6aa0e7046bb7baca0cc61443839a593fce5b4e801261409b0bf97ff160feb3" exitCode=0 Sep 09 06:46:43.529157 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:43.528758 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d976b9bf8-nxpnt" Sep 09 06:46:43.529157 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:43.528780 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d976b9bf8-nxpnt" event={"ID":"0c2234cd-5566-460a-ad8e-7cbbf25f805e","Type":"ContainerDied","Data":"ce6aa0e7046bb7baca0cc61443839a593fce5b4e801261409b0bf97ff160feb3"} Sep 09 06:46:43.529157 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:43.528837 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d976b9bf8-nxpnt" event={"ID":"0c2234cd-5566-460a-ad8e-7cbbf25f805e","Type":"ContainerDied","Data":"feb194b2cc55092522711e3cd8339da75099ffca0dbf5bbdba09f9f48df61434"} Sep 09 06:46:43.529157 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:43.528865 2559 scope.go:117] "RemoveContainer" containerID="ce6aa0e7046bb7baca0cc61443839a593fce5b4e801261409b0bf97ff160feb3" Sep 09 06:46:43.538374 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:43.538347 2559 scope.go:117] "RemoveContainer" containerID="ce6aa0e7046bb7baca0cc61443839a593fce5b4e801261409b0bf97ff160feb3" Sep 09 06:46:43.538660 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:46:43.538641 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce6aa0e7046bb7baca0cc61443839a593fce5b4e801261409b0bf97ff160feb3\": container with ID starting with ce6aa0e7046bb7baca0cc61443839a593fce5b4e801261409b0bf97ff160feb3 not found: ID does not exist" containerID="ce6aa0e7046bb7baca0cc61443839a593fce5b4e801261409b0bf97ff160feb3" Sep 09 06:46:43.538730 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:43.538675 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce6aa0e7046bb7baca0cc61443839a593fce5b4e801261409b0bf97ff160feb3"} err="failed to get container status \"ce6aa0e7046bb7baca0cc61443839a593fce5b4e801261409b0bf97ff160feb3\": rpc error: code = NotFound desc = could not find container \"ce6aa0e7046bb7baca0cc61443839a593fce5b4e801261409b0bf97ff160feb3\": container with ID starting with ce6aa0e7046bb7baca0cc61443839a593fce5b4e801261409b0bf97ff160feb3 not found: ID does not exist" Sep 09 06:46:43.562524 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:43.562491 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d976b9bf8-nxpnt"] Sep 09 06:46:43.571266 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:43.571242 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6d976b9bf8-nxpnt"] Sep 09 06:46:44.701976 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:46:44.701939 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c2234cd-5566-460a-ad8e-7cbbf25f805e" path="/var/lib/kubelet/pods/0c2234cd-5566-460a-ad8e-7cbbf25f805e/volumes" Sep 09 06:48:22.499654 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:22.499625 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nf9gf_26d28788-4f98-454b-ab39-2d0d857aa674/console-operator/2.log" Sep 09 06:48:22.500374 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:22.500349 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nf9gf_26d28788-4f98-454b-ab39-2d0d857aa674/console-operator/2.log" Sep 09 06:48:40.705654 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:40.705620 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-98b9c5856-7bk87"] Sep 09 06:48:40.707979 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:40.705925 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c2234cd-5566-460a-ad8e-7cbbf25f805e" containerName="console" Sep 09 06:48:40.707979 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:40.705936 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c2234cd-5566-460a-ad8e-7cbbf25f805e" containerName="console" Sep 09 06:48:40.707979 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:40.706003 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c2234cd-5566-460a-ad8e-7cbbf25f805e" containerName="console" Sep 09 06:48:40.708846 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:40.708831 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-98b9c5856-7bk87" Sep 09 06:48:40.711167 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:40.711143 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Sep 09 06:48:40.711935 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:40.711918 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Sep 09 06:48:40.712023 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:40.711954 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-gdnlx\"" Sep 09 06:48:40.724888 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:40.724870 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-98b9c5856-7bk87"] Sep 09 06:48:40.866944 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:40.866913 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssz2k\" (UniqueName: \"kubernetes.io/projected/247e9f69-e45c-49e7-be80-42c2ad4e0e0f-kube-api-access-ssz2k\") pod \"feast-operator-controller-manager-98b9c5856-7bk87\" (UID: \"247e9f69-e45c-49e7-be80-42c2ad4e0e0f\") " pod="feast-operator-system/feast-operator-controller-manager-98b9c5856-7bk87" Sep 09 06:48:40.967481 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:40.967383 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssz2k\" (UniqueName: \"kubernetes.io/projected/247e9f69-e45c-49e7-be80-42c2ad4e0e0f-kube-api-access-ssz2k\") pod \"feast-operator-controller-manager-98b9c5856-7bk87\" (UID: \"247e9f69-e45c-49e7-be80-42c2ad4e0e0f\") " pod="feast-operator-system/feast-operator-controller-manager-98b9c5856-7bk87" Sep 09 06:48:40.975458 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:40.975426 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssz2k\" (UniqueName: \"kubernetes.io/projected/247e9f69-e45c-49e7-be80-42c2ad4e0e0f-kube-api-access-ssz2k\") pod \"feast-operator-controller-manager-98b9c5856-7bk87\" (UID: \"247e9f69-e45c-49e7-be80-42c2ad4e0e0f\") " pod="feast-operator-system/feast-operator-controller-manager-98b9c5856-7bk87" Sep 09 06:48:41.019483 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:41.019451 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-98b9c5856-7bk87" Sep 09 06:48:41.291782 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:41.291702 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-98b9c5856-7bk87"] Sep 09 06:48:41.295696 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:48:41.295669 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod247e9f69_e45c_49e7_be80_42c2ad4e0e0f.slice/crio-cea0fa9cb7d11c678902b7bde86a312362d0e6c38e83c8db2c302b66d3c2b0cf WatchSource:0}: Error finding container cea0fa9cb7d11c678902b7bde86a312362d0e6c38e83c8db2c302b66d3c2b0cf: Status 404 returned error can't find the container with id cea0fa9cb7d11c678902b7bde86a312362d0e6c38e83c8db2c302b66d3c2b0cf Sep 09 06:48:41.297597 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:41.297579 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 06:48:42.016791 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:42.016749 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-98b9c5856-7bk87" event={"ID":"247e9f69-e45c-49e7-be80-42c2ad4e0e0f","Type":"ContainerStarted","Data":"cea0fa9cb7d11c678902b7bde86a312362d0e6c38e83c8db2c302b66d3c2b0cf"} Sep 09 06:48:44.027622 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:44.027583 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-98b9c5856-7bk87" event={"ID":"247e9f69-e45c-49e7-be80-42c2ad4e0e0f","Type":"ContainerStarted","Data":"324c40df5415380b5e02c3e967557dc90d655802556450aa57e322ec753afaa8"} Sep 09 06:48:44.027997 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:44.027777 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-98b9c5856-7bk87" Sep 09 06:48:44.053451 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:44.053405 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-98b9c5856-7bk87" podStartSLOduration=1.746837454 podStartE2EDuration="4.053389533s" totalImagesPullingTime="2.306552079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:48:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:48:44.053200491 +0000 UTC m=+621.949617913" watchObservedRunningTime="2026-09-09 06:48:44.053389533 +0000 UTC m=+621.949806954" Sep 09 06:48:46.253511 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:46.253427 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-69f4dbf49b-j8kmp"] Sep 09 06:48:46.256995 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:46.256969 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69f4dbf49b-j8kmp" Sep 09 06:48:46.269602 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:46.269577 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69f4dbf49b-j8kmp"] Sep 09 06:48:46.411907 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:46.411870 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/714ac9f9-377b-4a71-8824-b6e8a1ca225b-service-ca\") pod \"console-69f4dbf49b-j8kmp\" (UID: \"714ac9f9-377b-4a71-8824-b6e8a1ca225b\") " pod="openshift-console/console-69f4dbf49b-j8kmp" Sep 09 06:48:46.411907 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:46.411908 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/714ac9f9-377b-4a71-8824-b6e8a1ca225b-trusted-ca-bundle\") pod \"console-69f4dbf49b-j8kmp\" (UID: \"714ac9f9-377b-4a71-8824-b6e8a1ca225b\") " pod="openshift-console/console-69f4dbf49b-j8kmp" Sep 09 06:48:46.412186 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:46.411947 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/714ac9f9-377b-4a71-8824-b6e8a1ca225b-console-oauth-config\") pod \"console-69f4dbf49b-j8kmp\" (UID: \"714ac9f9-377b-4a71-8824-b6e8a1ca225b\") " pod="openshift-console/console-69f4dbf49b-j8kmp" Sep 09 06:48:46.412186 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:46.412027 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rxhc\" (UniqueName: \"kubernetes.io/projected/714ac9f9-377b-4a71-8824-b6e8a1ca225b-kube-api-access-6rxhc\") pod \"console-69f4dbf49b-j8kmp\" (UID: \"714ac9f9-377b-4a71-8824-b6e8a1ca225b\") " pod="openshift-console/console-69f4dbf49b-j8kmp" Sep 09 06:48:46.412186 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:46.412094 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/714ac9f9-377b-4a71-8824-b6e8a1ca225b-oauth-serving-cert\") pod \"console-69f4dbf49b-j8kmp\" (UID: \"714ac9f9-377b-4a71-8824-b6e8a1ca225b\") " pod="openshift-console/console-69f4dbf49b-j8kmp" Sep 09 06:48:46.412186 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:46.412150 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/714ac9f9-377b-4a71-8824-b6e8a1ca225b-console-serving-cert\") pod \"console-69f4dbf49b-j8kmp\" (UID: \"714ac9f9-377b-4a71-8824-b6e8a1ca225b\") " pod="openshift-console/console-69f4dbf49b-j8kmp" Sep 09 06:48:46.412341 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:46.412199 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/714ac9f9-377b-4a71-8824-b6e8a1ca225b-console-config\") pod \"console-69f4dbf49b-j8kmp\" (UID: \"714ac9f9-377b-4a71-8824-b6e8a1ca225b\") " pod="openshift-console/console-69f4dbf49b-j8kmp" Sep 09 06:48:46.513337 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:46.513255 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/714ac9f9-377b-4a71-8824-b6e8a1ca225b-service-ca\") pod \"console-69f4dbf49b-j8kmp\" (UID: \"714ac9f9-377b-4a71-8824-b6e8a1ca225b\") " pod="openshift-console/console-69f4dbf49b-j8kmp" Sep 09 06:48:46.513337 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:46.513289 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/714ac9f9-377b-4a71-8824-b6e8a1ca225b-trusted-ca-bundle\") pod \"console-69f4dbf49b-j8kmp\" (UID: \"714ac9f9-377b-4a71-8824-b6e8a1ca225b\") " pod="openshift-console/console-69f4dbf49b-j8kmp" Sep 09 06:48:46.513337 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:46.513318 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/714ac9f9-377b-4a71-8824-b6e8a1ca225b-console-oauth-config\") pod \"console-69f4dbf49b-j8kmp\" (UID: \"714ac9f9-377b-4a71-8824-b6e8a1ca225b\") " pod="openshift-console/console-69f4dbf49b-j8kmp" Sep 09 06:48:46.513633 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:46.513342 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rxhc\" (UniqueName: \"kubernetes.io/projected/714ac9f9-377b-4a71-8824-b6e8a1ca225b-kube-api-access-6rxhc\") pod \"console-69f4dbf49b-j8kmp\" (UID: \"714ac9f9-377b-4a71-8824-b6e8a1ca225b\") " pod="openshift-console/console-69f4dbf49b-j8kmp" Sep 09 06:48:46.513633 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:46.513364 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/714ac9f9-377b-4a71-8824-b6e8a1ca225b-oauth-serving-cert\") pod \"console-69f4dbf49b-j8kmp\" (UID: \"714ac9f9-377b-4a71-8824-b6e8a1ca225b\") " pod="openshift-console/console-69f4dbf49b-j8kmp" Sep 09 06:48:46.513633 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:46.513401 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/714ac9f9-377b-4a71-8824-b6e8a1ca225b-console-serving-cert\") pod \"console-69f4dbf49b-j8kmp\" (UID: \"714ac9f9-377b-4a71-8824-b6e8a1ca225b\") " pod="openshift-console/console-69f4dbf49b-j8kmp" Sep 09 06:48:46.513633 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:46.513449 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/714ac9f9-377b-4a71-8824-b6e8a1ca225b-console-config\") pod \"console-69f4dbf49b-j8kmp\" (UID: \"714ac9f9-377b-4a71-8824-b6e8a1ca225b\") " pod="openshift-console/console-69f4dbf49b-j8kmp" Sep 09 06:48:46.514248 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:46.514220 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/714ac9f9-377b-4a71-8824-b6e8a1ca225b-service-ca\") pod \"console-69f4dbf49b-j8kmp\" (UID: \"714ac9f9-377b-4a71-8824-b6e8a1ca225b\") " pod="openshift-console/console-69f4dbf49b-j8kmp" Sep 09 06:48:46.514248 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:46.514232 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/714ac9f9-377b-4a71-8824-b6e8a1ca225b-oauth-serving-cert\") pod \"console-69f4dbf49b-j8kmp\" (UID: \"714ac9f9-377b-4a71-8824-b6e8a1ca225b\") " pod="openshift-console/console-69f4dbf49b-j8kmp" Sep 09 06:48:46.514388 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:46.514239 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/714ac9f9-377b-4a71-8824-b6e8a1ca225b-console-config\") pod \"console-69f4dbf49b-j8kmp\" (UID: \"714ac9f9-377b-4a71-8824-b6e8a1ca225b\") " pod="openshift-console/console-69f4dbf49b-j8kmp" Sep 09 06:48:46.514388 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:46.514236 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/714ac9f9-377b-4a71-8824-b6e8a1ca225b-trusted-ca-bundle\") pod \"console-69f4dbf49b-j8kmp\" (UID: \"714ac9f9-377b-4a71-8824-b6e8a1ca225b\") " pod="openshift-console/console-69f4dbf49b-j8kmp" Sep 09 06:48:46.516340 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:46.516313 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/714ac9f9-377b-4a71-8824-b6e8a1ca225b-console-serving-cert\") pod \"console-69f4dbf49b-j8kmp\" (UID: \"714ac9f9-377b-4a71-8824-b6e8a1ca225b\") " pod="openshift-console/console-69f4dbf49b-j8kmp" Sep 09 06:48:46.516438 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:46.516417 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/714ac9f9-377b-4a71-8824-b6e8a1ca225b-console-oauth-config\") pod \"console-69f4dbf49b-j8kmp\" (UID: \"714ac9f9-377b-4a71-8824-b6e8a1ca225b\") " pod="openshift-console/console-69f4dbf49b-j8kmp" Sep 09 06:48:46.522038 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:46.522015 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rxhc\" (UniqueName: \"kubernetes.io/projected/714ac9f9-377b-4a71-8824-b6e8a1ca225b-kube-api-access-6rxhc\") pod \"console-69f4dbf49b-j8kmp\" (UID: \"714ac9f9-377b-4a71-8824-b6e8a1ca225b\") " pod="openshift-console/console-69f4dbf49b-j8kmp" Sep 09 06:48:46.567706 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:46.567683 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69f4dbf49b-j8kmp" Sep 09 06:48:46.840685 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:46.840659 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69f4dbf49b-j8kmp"] Sep 09 06:48:46.843287 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:48:46.843260 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod714ac9f9_377b_4a71_8824_b6e8a1ca225b.slice/crio-cdfa734409e0d0685226c4b22edea58bebe251ccbfc8db6f47009434c6295ecd WatchSource:0}: Error finding container cdfa734409e0d0685226c4b22edea58bebe251ccbfc8db6f47009434c6295ecd: Status 404 returned error can't find the container with id cdfa734409e0d0685226c4b22edea58bebe251ccbfc8db6f47009434c6295ecd Sep 09 06:48:47.042219 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:47.042184 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69f4dbf49b-j8kmp" event={"ID":"714ac9f9-377b-4a71-8824-b6e8a1ca225b","Type":"ContainerStarted","Data":"af9bf2c64170018411f994d58f87ee5e604fc9cc59dd11625e39ed34fbb05b84"} Sep 09 06:48:47.042381 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:47.042234 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69f4dbf49b-j8kmp" event={"ID":"714ac9f9-377b-4a71-8824-b6e8a1ca225b","Type":"ContainerStarted","Data":"cdfa734409e0d0685226c4b22edea58bebe251ccbfc8db6f47009434c6295ecd"} Sep 09 06:48:47.064902 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:47.064854 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-69f4dbf49b-j8kmp" podStartSLOduration=1.064838939 podStartE2EDuration="1.064838939s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:48:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:48:47.063428862 +0000 UTC m=+624.959846285" watchObservedRunningTime="2026-09-09 06:48:47.064838939 +0000 UTC m=+624.961256360" Sep 09 06:48:55.034362 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:55.034326 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-98b9c5856-7bk87" Sep 09 06:48:56.568192 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:56.568156 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-69f4dbf49b-j8kmp" Sep 09 06:48:56.568589 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:56.568247 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-69f4dbf49b-j8kmp" Sep 09 06:48:56.572962 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:56.572938 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-69f4dbf49b-j8kmp" Sep 09 06:48:57.094288 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:57.094257 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-69f4dbf49b-j8kmp" Sep 09 06:48:57.140129 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:48:57.140036 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b597d587b-g2vg8"] Sep 09 06:49:19.419770 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:19.419732 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh"] Sep 09 06:49:19.427570 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:19.427553 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:49:19.431478 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:19.431448 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 09 06:49:19.431478 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:19.431466 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 09 06:49:19.431685 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:19.431448 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 09 06:49:19.431685 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:19.431555 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-zdksg\"" Sep 09 06:49:19.431685 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:19.431560 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 09 06:49:19.431808 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:19.431740 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 09 06:49:19.431808 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:19.431793 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 09 06:49:19.431878 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:19.431840 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\"" Sep 09 06:49:19.435585 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:19.435104 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh"] Sep 09 06:49:19.473823 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:19.473798 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/bbc37446-f358-40b2-9dad-0041d189b76b-registry-tls\") pod \"feast-simple-feast-setup-84dccd7859-s6hgh\" (UID: \"bbc37446-f358-40b2-9dad-0041d189b76b\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:49:19.473935 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:19.473835 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bbc37446-f358-40b2-9dad-0041d189b76b-online-tls\") pod \"feast-simple-feast-setup-84dccd7859-s6hgh\" (UID: \"bbc37446-f358-40b2-9dad-0041d189b76b\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:49:19.473935 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:19.473866 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/bbc37446-f358-40b2-9dad-0041d189b76b-ui-tls\") pod \"feast-simple-feast-setup-84dccd7859-s6hgh\" (UID: \"bbc37446-f358-40b2-9dad-0041d189b76b\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:49:19.473935 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:19.473906 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bbc37446-f358-40b2-9dad-0041d189b76b-feast-data\") pod \"feast-simple-feast-setup-84dccd7859-s6hgh\" (UID: \"bbc37446-f358-40b2-9dad-0041d189b76b\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:49:19.474081 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:19.473960 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/bbc37446-f358-40b2-9dad-0041d189b76b-offline-tls\") pod \"feast-simple-feast-setup-84dccd7859-s6hgh\" (UID: \"bbc37446-f358-40b2-9dad-0041d189b76b\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:49:19.474081 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:19.473982 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx666\" (UniqueName: \"kubernetes.io/projected/bbc37446-f358-40b2-9dad-0041d189b76b-kube-api-access-rx666\") pod \"feast-simple-feast-setup-84dccd7859-s6hgh\" (UID: \"bbc37446-f358-40b2-9dad-0041d189b76b\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:49:19.574347 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:19.574319 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/bbc37446-f358-40b2-9dad-0041d189b76b-registry-tls\") pod \"feast-simple-feast-setup-84dccd7859-s6hgh\" (UID: \"bbc37446-f358-40b2-9dad-0041d189b76b\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:49:19.574486 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:19.574363 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bbc37446-f358-40b2-9dad-0041d189b76b-online-tls\") pod \"feast-simple-feast-setup-84dccd7859-s6hgh\" (UID: \"bbc37446-f358-40b2-9dad-0041d189b76b\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:49:19.574527 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:19.574487 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/bbc37446-f358-40b2-9dad-0041d189b76b-ui-tls\") pod \"feast-simple-feast-setup-84dccd7859-s6hgh\" (UID: \"bbc37446-f358-40b2-9dad-0041d189b76b\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:49:19.574573 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:19.574537 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bbc37446-f358-40b2-9dad-0041d189b76b-feast-data\") pod \"feast-simple-feast-setup-84dccd7859-s6hgh\" (UID: \"bbc37446-f358-40b2-9dad-0041d189b76b\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:49:19.574619 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:19.574601 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/bbc37446-f358-40b2-9dad-0041d189b76b-offline-tls\") pod \"feast-simple-feast-setup-84dccd7859-s6hgh\" (UID: \"bbc37446-f358-40b2-9dad-0041d189b76b\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:49:19.574670 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:49:19.574622 2559 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Sep 09 06:49:19.574670 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:19.574644 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx666\" (UniqueName: \"kubernetes.io/projected/bbc37446-f358-40b2-9dad-0041d189b76b-kube-api-access-rx666\") pod \"feast-simple-feast-setup-84dccd7859-s6hgh\" (UID: \"bbc37446-f358-40b2-9dad-0041d189b76b\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:49:19.574771 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:49:19.574679 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbc37446-f358-40b2-9dad-0041d189b76b-ui-tls podName:bbc37446-f358-40b2-9dad-0041d189b76b nodeName:}" failed. No retries permitted until 2026-09-09 06:49:20.074660205 +0000 UTC m=+657.971077605 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/bbc37446-f358-40b2-9dad-0041d189b76b-ui-tls") pod "feast-simple-feast-setup-84dccd7859-s6hgh" (UID: "bbc37446-f358-40b2-9dad-0041d189b76b") : secret "feast-simple-feast-setup-ui-tls" not found Sep 09 06:49:19.575049 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:19.575012 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bbc37446-f358-40b2-9dad-0041d189b76b-feast-data\") pod \"feast-simple-feast-setup-84dccd7859-s6hgh\" (UID: \"bbc37446-f358-40b2-9dad-0041d189b76b\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:49:19.576865 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:19.576836 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/bbc37446-f358-40b2-9dad-0041d189b76b-offline-tls\") pod \"feast-simple-feast-setup-84dccd7859-s6hgh\" (UID: \"bbc37446-f358-40b2-9dad-0041d189b76b\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:49:19.576971 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:19.576953 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bbc37446-f358-40b2-9dad-0041d189b76b-online-tls\") pod \"feast-simple-feast-setup-84dccd7859-s6hgh\" (UID: \"bbc37446-f358-40b2-9dad-0041d189b76b\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:49:19.577016 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:19.576995 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/bbc37446-f358-40b2-9dad-0041d189b76b-registry-tls\") pod \"feast-simple-feast-setup-84dccd7859-s6hgh\" (UID: \"bbc37446-f358-40b2-9dad-0041d189b76b\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:49:19.583115 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:19.583094 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx666\" (UniqueName: \"kubernetes.io/projected/bbc37446-f358-40b2-9dad-0041d189b76b-kube-api-access-rx666\") pod \"feast-simple-feast-setup-84dccd7859-s6hgh\" (UID: \"bbc37446-f358-40b2-9dad-0041d189b76b\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:49:20.079586 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:20.079551 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/bbc37446-f358-40b2-9dad-0041d189b76b-ui-tls\") pod \"feast-simple-feast-setup-84dccd7859-s6hgh\" (UID: \"bbc37446-f358-40b2-9dad-0041d189b76b\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:49:20.081984 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:20.081959 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/bbc37446-f358-40b2-9dad-0041d189b76b-ui-tls\") pod \"feast-simple-feast-setup-84dccd7859-s6hgh\" (UID: \"bbc37446-f358-40b2-9dad-0041d189b76b\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:49:20.344764 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:20.344687 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:49:20.623896 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:20.623871 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh"] Sep 09 06:49:20.626349 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:49:20.626320 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbbc37446_f358_40b2_9dad_0041d189b76b.slice/crio-21981ebca68c8e246aa076e236f178d7c66f240defa4cf888477e438743653e3 WatchSource:0}: Error finding container 21981ebca68c8e246aa076e236f178d7c66f240defa4cf888477e438743653e3: Status 404 returned error can't find the container with id 21981ebca68c8e246aa076e236f178d7c66f240defa4cf888477e438743653e3 Sep 09 06:49:21.201836 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:21.201803 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" event={"ID":"bbc37446-f358-40b2-9dad-0041d189b76b","Type":"ContainerStarted","Data":"21981ebca68c8e246aa076e236f178d7c66f240defa4cf888477e438743653e3"} Sep 09 06:49:22.163712 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:22.163670 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5b597d587b-g2vg8" podUID="e9a7cc46-4f97-4021-a031-b990097ff56e" containerName="console" containerID="cri-o://cb58326814cacb513ff234af577b2acde49054164275f79425c68a9a3938baec" gracePeriod=15 Sep 09 06:49:23.218251 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:23.218151 2559 generic.go:356] "Generic (PLEG): container finished" podID="e9a7cc46-4f97-4021-a031-b990097ff56e" containerID="cb58326814cacb513ff234af577b2acde49054164275f79425c68a9a3938baec" exitCode=0 Sep 09 06:49:23.218251 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:23.218205 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b597d587b-g2vg8" event={"ID":"e9a7cc46-4f97-4021-a031-b990097ff56e","Type":"ContainerDied","Data":"cb58326814cacb513ff234af577b2acde49054164275f79425c68a9a3938baec"} Sep 09 06:49:23.610107 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:23.610085 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b597d587b-g2vg8" Sep 09 06:49:23.711864 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:23.711829 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xw54q\" (UniqueName: \"kubernetes.io/projected/e9a7cc46-4f97-4021-a031-b990097ff56e-kube-api-access-xw54q\") pod \"e9a7cc46-4f97-4021-a031-b990097ff56e\" (UID: \"e9a7cc46-4f97-4021-a031-b990097ff56e\") " Sep 09 06:49:23.712043 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:23.711882 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e9a7cc46-4f97-4021-a031-b990097ff56e-oauth-serving-cert\") pod \"e9a7cc46-4f97-4021-a031-b990097ff56e\" (UID: \"e9a7cc46-4f97-4021-a031-b990097ff56e\") " Sep 09 06:49:23.712043 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:23.711909 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e9a7cc46-4f97-4021-a031-b990097ff56e-console-oauth-config\") pod \"e9a7cc46-4f97-4021-a031-b990097ff56e\" (UID: \"e9a7cc46-4f97-4021-a031-b990097ff56e\") " Sep 09 06:49:23.712043 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:23.711984 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9a7cc46-4f97-4021-a031-b990097ff56e-trusted-ca-bundle\") pod \"e9a7cc46-4f97-4021-a031-b990097ff56e\" (UID: \"e9a7cc46-4f97-4021-a031-b990097ff56e\") " Sep 09 06:49:23.712043 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:23.712010 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e9a7cc46-4f97-4021-a031-b990097ff56e-service-ca\") pod \"e9a7cc46-4f97-4021-a031-b990097ff56e\" (UID: \"e9a7cc46-4f97-4021-a031-b990097ff56e\") " Sep 09 06:49:23.712293 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:23.712080 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e9a7cc46-4f97-4021-a031-b990097ff56e-console-config\") pod \"e9a7cc46-4f97-4021-a031-b990097ff56e\" (UID: \"e9a7cc46-4f97-4021-a031-b990097ff56e\") " Sep 09 06:49:23.712293 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:23.712108 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e9a7cc46-4f97-4021-a031-b990097ff56e-console-serving-cert\") pod \"e9a7cc46-4f97-4021-a031-b990097ff56e\" (UID: \"e9a7cc46-4f97-4021-a031-b990097ff56e\") " Sep 09 06:49:23.712387 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:23.712360 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9a7cc46-4f97-4021-a031-b990097ff56e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e9a7cc46-4f97-4021-a031-b990097ff56e" (UID: "e9a7cc46-4f97-4021-a031-b990097ff56e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 06:49:23.712615 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:23.712583 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9a7cc46-4f97-4021-a031-b990097ff56e-service-ca" (OuterVolumeSpecName: "service-ca") pod "e9a7cc46-4f97-4021-a031-b990097ff56e" (UID: "e9a7cc46-4f97-4021-a031-b990097ff56e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 06:49:23.712615 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:23.712592 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9a7cc46-4f97-4021-a031-b990097ff56e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e9a7cc46-4f97-4021-a031-b990097ff56e" (UID: "e9a7cc46-4f97-4021-a031-b990097ff56e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 06:49:23.712797 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:23.712598 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9a7cc46-4f97-4021-a031-b990097ff56e-console-config" (OuterVolumeSpecName: "console-config") pod "e9a7cc46-4f97-4021-a031-b990097ff56e" (UID: "e9a7cc46-4f97-4021-a031-b990097ff56e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 06:49:23.714548 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:23.714520 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9a7cc46-4f97-4021-a031-b990097ff56e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e9a7cc46-4f97-4021-a031-b990097ff56e" (UID: "e9a7cc46-4f97-4021-a031-b990097ff56e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 06:49:23.714658 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:23.714600 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9a7cc46-4f97-4021-a031-b990097ff56e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e9a7cc46-4f97-4021-a031-b990097ff56e" (UID: "e9a7cc46-4f97-4021-a031-b990097ff56e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 06:49:23.714658 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:23.714613 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9a7cc46-4f97-4021-a031-b990097ff56e-kube-api-access-xw54q" (OuterVolumeSpecName: "kube-api-access-xw54q") pod "e9a7cc46-4f97-4021-a031-b990097ff56e" (UID: "e9a7cc46-4f97-4021-a031-b990097ff56e"). InnerVolumeSpecName "kube-api-access-xw54q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 06:49:23.813093 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:23.812987 2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9a7cc46-4f97-4021-a031-b990097ff56e-trusted-ca-bundle\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:49:23.813093 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:23.813035 2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e9a7cc46-4f97-4021-a031-b990097ff56e-service-ca\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:49:23.813093 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:23.813062 2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e9a7cc46-4f97-4021-a031-b990097ff56e-console-config\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:49:23.813093 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:23.813078 2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e9a7cc46-4f97-4021-a031-b990097ff56e-console-serving-cert\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:49:23.813093 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:23.813091 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xw54q\" (UniqueName: \"kubernetes.io/projected/e9a7cc46-4f97-4021-a031-b990097ff56e-kube-api-access-xw54q\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:49:23.813399 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:23.813105 2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e9a7cc46-4f97-4021-a031-b990097ff56e-oauth-serving-cert\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:49:23.813399 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:23.813148 2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e9a7cc46-4f97-4021-a031-b990097ff56e-console-oauth-config\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:49:24.225201 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:24.225176 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b597d587b-g2vg8" Sep 09 06:49:24.225648 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:24.225208 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b597d587b-g2vg8" event={"ID":"e9a7cc46-4f97-4021-a031-b990097ff56e","Type":"ContainerDied","Data":"4d74b4da827a1dd0e687061d1b758d9a9d6c47dbeb275f0670468f96f8387ac1"} Sep 09 06:49:24.225648 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:24.225282 2559 scope.go:117] "RemoveContainer" containerID="cb58326814cacb513ff234af577b2acde49054164275f79425c68a9a3938baec" Sep 09 06:49:24.249711 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:24.249682 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b597d587b-g2vg8"] Sep 09 06:49:24.253749 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:24.253726 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5b597d587b-g2vg8"] Sep 09 06:49:24.703455 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:24.703414 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9a7cc46-4f97-4021-a031-b990097ff56e" path="/var/lib/kubelet/pods/e9a7cc46-4f97-4021-a031-b990097ff56e/volumes" Sep 09 06:49:45.330410 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:45.330375 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" event={"ID":"bbc37446-f358-40b2-9dad-0041d189b76b","Type":"ContainerStarted","Data":"cf609aec6f863a485462cddb820140b0b0bdd86ce0c7c7d3749cfa25adbfdcfc"} Sep 09 06:49:49.349211 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:49.349178 2559 generic.go:356] "Generic (PLEG): container finished" podID="bbc37446-f358-40b2-9dad-0041d189b76b" containerID="cf609aec6f863a485462cddb820140b0b0bdd86ce0c7c7d3749cfa25adbfdcfc" exitCode=0 Sep 09 06:49:49.349607 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:49.349251 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" event={"ID":"bbc37446-f358-40b2-9dad-0041d189b76b","Type":"ContainerDied","Data":"cf609aec6f863a485462cddb820140b0b0bdd86ce0c7c7d3749cfa25adbfdcfc"} Sep 09 06:49:50.357938 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:50.357903 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" event={"ID":"bbc37446-f358-40b2-9dad-0041d189b76b","Type":"ContainerStarted","Data":"b099d479f742dddd0de7de5697605602104f8a489d34cb0219a92bc09e078193"} Sep 09 06:49:55.381215 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:55.381135 2559 generic.go:356] "Generic (PLEG): container finished" podID="bbc37446-f358-40b2-9dad-0041d189b76b" containerID="b099d479f742dddd0de7de5697605602104f8a489d34cb0219a92bc09e078193" exitCode=0 Sep 09 06:49:55.381597 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:55.381208 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" event={"ID":"bbc37446-f358-40b2-9dad-0041d189b76b","Type":"ContainerDied","Data":"b099d479f742dddd0de7de5697605602104f8a489d34cb0219a92bc09e078193"} Sep 09 06:49:55.390025 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:55.390004 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\"" Sep 09 06:49:56.392923 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:56.392877 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" event={"ID":"bbc37446-f358-40b2-9dad-0041d189b76b","Type":"ContainerStarted","Data":"0c3d797079871daa8e9c4ce5c01ac989790a9579a4dcfb9776f46a09e5f5135b"} Sep 09 06:49:56.393477 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:56.392939 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" event={"ID":"bbc37446-f358-40b2-9dad-0041d189b76b","Type":"ContainerStarted","Data":"64a63363668f558234bdc6e1ce5654bcdf4c245c5fc64de7938bd164fb19bb52"} Sep 09 06:49:56.393477 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:56.392959 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" event={"ID":"bbc37446-f358-40b2-9dad-0041d189b76b","Type":"ContainerStarted","Data":"18edd35f88dee8daa0496921fc0b3a44c1856696ca5da28f5843c57f41b5f54f"} Sep 09 06:49:56.393477 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:56.392977 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" event={"ID":"bbc37446-f358-40b2-9dad-0041d189b76b","Type":"ContainerStarted","Data":"03c8438c2669c502c613bfc2d12a2467eadfd500cfab76ecd9fc6f1d06187324"} Sep 09 06:49:59.345562 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:59.345523 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:49:59.346036 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:59.345594 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:49:59.346036 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:59.345615 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:49:59.346036 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:59.345631 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:49:59.347664 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:59.347621 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" podUID="bbc37446-f358-40b2-9dad-0041d189b76b" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.30:6571: connect: connection refused" Sep 09 06:49:59.347792 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:59.347733 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" podUID="bbc37446-f358-40b2-9dad-0041d189b76b" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.30:8816: connect: connection refused" Sep 09 06:49:59.347854 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:59.347815 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" podUID="bbc37446-f358-40b2-9dad-0041d189b76b" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.30:8443: connect: connection refused" Sep 09 06:49:59.347946 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:49:59.347925 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" podUID="bbc37446-f358-40b2-9dad-0041d189b76b" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": dial tcp 10.132.0.30:6567: connect: connection refused" Sep 09 06:50:00.345788 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:50:00.345746 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:50:00.346413 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:50:00.346384 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:50:00.346540 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:50:00.346424 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:50:00.346540 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:50:00.346445 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:50:02.345695 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:50:02.345648 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" podUID="bbc37446-f358-40b2-9dad-0041d189b76b" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.30:6571: connect: connection refused" Sep 09 06:50:02.346402 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:50:02.345651 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" podUID="bbc37446-f358-40b2-9dad-0041d189b76b" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.30:8443: connect: connection refused" Sep 09 06:50:02.346681 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:50:02.345648 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" podUID="bbc37446-f358-40b2-9dad-0041d189b76b" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.30:8816: connect: connection refused" Sep 09 06:50:02.346681 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:50:02.345721 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" podUID="bbc37446-f358-40b2-9dad-0041d189b76b" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": dial tcp 10.132.0.30:6567: connect: connection refused" Sep 09 06:50:05.345970 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:50:05.345932 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:50:05.346516 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:50:05.346488 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:50:05.346701 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:50:05.346686 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:50:05.346769 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:50:05.346757 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:50:05.351664 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:50:05.351638 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:50:05.372895 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:50:05.372852 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" podStartSLOduration=22.671009775 podStartE2EDuration="46.372835828s" totalImagesPullingTime="23.701826053s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:49:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:49:56.418249771 +0000 UTC m=+694.314667206" watchObservedRunningTime="2026-09-09 06:50:05.372835828 +0000 UTC m=+703.269253249" Sep 09 06:50:05.448767 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:50:05.448742 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:50:05.448886 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:50:05.448802 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:50:05.451268 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:50:05.451250 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:51:07.920783 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:07.920744 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh"] Sep 09 06:51:07.921340 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:07.921088 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" podUID="bbc37446-f358-40b2-9dad-0041d189b76b" containerName="registry" containerID="cri-o://03c8438c2669c502c613bfc2d12a2467eadfd500cfab76ecd9fc6f1d06187324" gracePeriod=30 Sep 09 06:51:07.921340 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:07.921137 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" podUID="bbc37446-f358-40b2-9dad-0041d189b76b" containerName="offline" containerID="cri-o://64a63363668f558234bdc6e1ce5654bcdf4c245c5fc64de7938bd164fb19bb52" gracePeriod=30 Sep 09 06:51:07.921340 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:07.921160 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" podUID="bbc37446-f358-40b2-9dad-0041d189b76b" containerName="online" containerID="cri-o://18edd35f88dee8daa0496921fc0b3a44c1856696ca5da28f5843c57f41b5f54f" gracePeriod=30 Sep 09 06:51:07.921340 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:07.921169 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" podUID="bbc37446-f358-40b2-9dad-0041d189b76b" containerName="ui" containerID="cri-o://0c3d797079871daa8e9c4ce5c01ac989790a9579a4dcfb9776f46a09e5f5135b" gracePeriod=30 Sep 09 06:51:08.302069 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:08.302020 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp"] Sep 09 06:51:08.302421 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:08.302408 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9a7cc46-4f97-4021-a031-b990097ff56e" containerName="console" Sep 09 06:51:08.302477 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:08.302423 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9a7cc46-4f97-4021-a031-b990097ff56e" containerName="console" Sep 09 06:51:08.302523 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:08.302487 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9a7cc46-4f97-4021-a031-b990097ff56e" containerName="console" Sep 09 06:51:08.306181 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:08.306162 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:51:08.308702 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:08.308683 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-548zx\"" Sep 09 06:51:08.323429 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:08.323401 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp"] Sep 09 06:51:08.402653 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:08.402615 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a0305aa7-15c7-4cb8-8d49-96e4437444a6-feast-data\") pod \"feast-simple-feast-setup-84dccd7859-r7kgp\" (UID: \"a0305aa7-15c7-4cb8-8d49-96e4437444a6\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:51:08.402653 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:08.402657 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a0305aa7-15c7-4cb8-8d49-96e4437444a6-offline-tls\") pod \"feast-simple-feast-setup-84dccd7859-r7kgp\" (UID: \"a0305aa7-15c7-4cb8-8d49-96e4437444a6\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:51:08.402896 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:08.402676 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a0305aa7-15c7-4cb8-8d49-96e4437444a6-ui-tls\") pod \"feast-simple-feast-setup-84dccd7859-r7kgp\" (UID: \"a0305aa7-15c7-4cb8-8d49-96e4437444a6\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:51:08.402896 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:08.402704 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a0305aa7-15c7-4cb8-8d49-96e4437444a6-online-tls\") pod \"feast-simple-feast-setup-84dccd7859-r7kgp\" (UID: \"a0305aa7-15c7-4cb8-8d49-96e4437444a6\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:51:08.402896 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:08.402782 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k56s4\" (UniqueName: \"kubernetes.io/projected/a0305aa7-15c7-4cb8-8d49-96e4437444a6-kube-api-access-k56s4\") pod \"feast-simple-feast-setup-84dccd7859-r7kgp\" (UID: \"a0305aa7-15c7-4cb8-8d49-96e4437444a6\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:51:08.403007 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:08.402921 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a0305aa7-15c7-4cb8-8d49-96e4437444a6-registry-tls\") pod \"feast-simple-feast-setup-84dccd7859-r7kgp\" (UID: \"a0305aa7-15c7-4cb8-8d49-96e4437444a6\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:51:08.503734 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:08.503695 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a0305aa7-15c7-4cb8-8d49-96e4437444a6-online-tls\") pod \"feast-simple-feast-setup-84dccd7859-r7kgp\" (UID: \"a0305aa7-15c7-4cb8-8d49-96e4437444a6\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:51:08.503908 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:08.503740 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k56s4\" (UniqueName: \"kubernetes.io/projected/a0305aa7-15c7-4cb8-8d49-96e4437444a6-kube-api-access-k56s4\") pod \"feast-simple-feast-setup-84dccd7859-r7kgp\" (UID: \"a0305aa7-15c7-4cb8-8d49-96e4437444a6\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:51:08.503908 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:51:08.503870 2559 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found Sep 09 06:51:08.504022 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:08.503903 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a0305aa7-15c7-4cb8-8d49-96e4437444a6-registry-tls\") pod \"feast-simple-feast-setup-84dccd7859-r7kgp\" (UID: \"a0305aa7-15c7-4cb8-8d49-96e4437444a6\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:51:08.504022 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:51:08.503962 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0305aa7-15c7-4cb8-8d49-96e4437444a6-online-tls podName:a0305aa7-15c7-4cb8-8d49-96e4437444a6 nodeName:}" failed. No retries permitted until 2026-09-09 06:51:09.003940175 +0000 UTC m=+766.900357578 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/a0305aa7-15c7-4cb8-8d49-96e4437444a6-online-tls") pod "feast-simple-feast-setup-84dccd7859-r7kgp" (UID: "a0305aa7-15c7-4cb8-8d49-96e4437444a6") : secret "feast-simple-feast-setup-online-tls" not found Sep 09 06:51:08.504022 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:51:08.504005 2559 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Sep 09 06:51:08.504287 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:08.504039 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a0305aa7-15c7-4cb8-8d49-96e4437444a6-feast-data\") pod \"feast-simple-feast-setup-84dccd7859-r7kgp\" (UID: \"a0305aa7-15c7-4cb8-8d49-96e4437444a6\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:51:08.504287 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:51:08.504077 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0305aa7-15c7-4cb8-8d49-96e4437444a6-registry-tls podName:a0305aa7-15c7-4cb8-8d49-96e4437444a6 nodeName:}" failed. No retries permitted until 2026-09-09 06:51:09.004042822 +0000 UTC m=+766.900460242 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/a0305aa7-15c7-4cb8-8d49-96e4437444a6-registry-tls") pod "feast-simple-feast-setup-84dccd7859-r7kgp" (UID: "a0305aa7-15c7-4cb8-8d49-96e4437444a6") : secret "feast-simple-feast-setup-registry-tls" not found Sep 09 06:51:08.504287 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:08.504115 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a0305aa7-15c7-4cb8-8d49-96e4437444a6-offline-tls\") pod \"feast-simple-feast-setup-84dccd7859-r7kgp\" (UID: \"a0305aa7-15c7-4cb8-8d49-96e4437444a6\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:51:08.504287 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:08.504139 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a0305aa7-15c7-4cb8-8d49-96e4437444a6-ui-tls\") pod \"feast-simple-feast-setup-84dccd7859-r7kgp\" (UID: \"a0305aa7-15c7-4cb8-8d49-96e4437444a6\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:51:08.504474 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:08.504453 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a0305aa7-15c7-4cb8-8d49-96e4437444a6-feast-data\") pod \"feast-simple-feast-setup-84dccd7859-r7kgp\" (UID: \"a0305aa7-15c7-4cb8-8d49-96e4437444a6\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:51:08.506686 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:08.506661 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a0305aa7-15c7-4cb8-8d49-96e4437444a6-offline-tls\") pod \"feast-simple-feast-setup-84dccd7859-r7kgp\" (UID: \"a0305aa7-15c7-4cb8-8d49-96e4437444a6\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:51:08.506801 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:08.506725 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a0305aa7-15c7-4cb8-8d49-96e4437444a6-ui-tls\") pod \"feast-simple-feast-setup-84dccd7859-r7kgp\" (UID: \"a0305aa7-15c7-4cb8-8d49-96e4437444a6\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:51:08.515660 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:08.515638 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k56s4\" (UniqueName: \"kubernetes.io/projected/a0305aa7-15c7-4cb8-8d49-96e4437444a6-kube-api-access-k56s4\") pod \"feast-simple-feast-setup-84dccd7859-r7kgp\" (UID: \"a0305aa7-15c7-4cb8-8d49-96e4437444a6\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:51:08.719302 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:08.719274 2559 generic.go:356] "Generic (PLEG): container finished" podID="bbc37446-f358-40b2-9dad-0041d189b76b" containerID="0c3d797079871daa8e9c4ce5c01ac989790a9579a4dcfb9776f46a09e5f5135b" exitCode=0 Sep 09 06:51:08.719434 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:08.719316 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" event={"ID":"bbc37446-f358-40b2-9dad-0041d189b76b","Type":"ContainerDied","Data":"0c3d797079871daa8e9c4ce5c01ac989790a9579a4dcfb9776f46a09e5f5135b"} Sep 09 06:51:09.008714 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:09.008621 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a0305aa7-15c7-4cb8-8d49-96e4437444a6-online-tls\") pod \"feast-simple-feast-setup-84dccd7859-r7kgp\" (UID: \"a0305aa7-15c7-4cb8-8d49-96e4437444a6\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:51:09.008714 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:09.008675 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a0305aa7-15c7-4cb8-8d49-96e4437444a6-registry-tls\") pod \"feast-simple-feast-setup-84dccd7859-r7kgp\" (UID: \"a0305aa7-15c7-4cb8-8d49-96e4437444a6\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:51:09.011038 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:09.011015 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a0305aa7-15c7-4cb8-8d49-96e4437444a6-registry-tls\") pod \"feast-simple-feast-setup-84dccd7859-r7kgp\" (UID: \"a0305aa7-15c7-4cb8-8d49-96e4437444a6\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:51:09.011118 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:09.011088 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a0305aa7-15c7-4cb8-8d49-96e4437444a6-online-tls\") pod \"feast-simple-feast-setup-84dccd7859-r7kgp\" (UID: \"a0305aa7-15c7-4cb8-8d49-96e4437444a6\") " pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:51:09.218235 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:09.218201 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:51:09.501772 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:09.501748 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp"] Sep 09 06:51:09.503420 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:51:09.503393 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda0305aa7_15c7_4cb8_8d49_96e4437444a6.slice/crio-025a9ffe435d8bbdfd6f205e9e67d0bc61bfa9e479ae8e8224782e82bea36ec2 WatchSource:0}: Error finding container 025a9ffe435d8bbdfd6f205e9e67d0bc61bfa9e479ae8e8224782e82bea36ec2: Status 404 returned error can't find the container with id 025a9ffe435d8bbdfd6f205e9e67d0bc61bfa9e479ae8e8224782e82bea36ec2 Sep 09 06:51:09.727351 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:09.727315 2559 generic.go:356] "Generic (PLEG): container finished" podID="bbc37446-f358-40b2-9dad-0041d189b76b" containerID="18edd35f88dee8daa0496921fc0b3a44c1856696ca5da28f5843c57f41b5f54f" exitCode=0 Sep 09 06:51:09.727532 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:09.727395 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" event={"ID":"bbc37446-f358-40b2-9dad-0041d189b76b","Type":"ContainerDied","Data":"18edd35f88dee8daa0496921fc0b3a44c1856696ca5da28f5843c57f41b5f54f"} Sep 09 06:51:09.729120 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:09.729092 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" event={"ID":"a0305aa7-15c7-4cb8-8d49-96e4437444a6","Type":"ContainerStarted","Data":"c966d7dc4798f2105ef58e5c73c69201475903283fe96b233f248f0b6e9e69a3"} Sep 09 06:51:09.729229 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:09.729133 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" event={"ID":"a0305aa7-15c7-4cb8-8d49-96e4437444a6","Type":"ContainerStarted","Data":"025a9ffe435d8bbdfd6f205e9e67d0bc61bfa9e479ae8e8224782e82bea36ec2"} Sep 09 06:51:14.752427 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:14.752394 2559 generic.go:356] "Generic (PLEG): container finished" podID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerID="c966d7dc4798f2105ef58e5c73c69201475903283fe96b233f248f0b6e9e69a3" exitCode=0 Sep 09 06:51:14.752840 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:14.752470 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" event={"ID":"a0305aa7-15c7-4cb8-8d49-96e4437444a6","Type":"ContainerDied","Data":"c966d7dc4798f2105ef58e5c73c69201475903283fe96b233f248f0b6e9e69a3"} Sep 09 06:51:15.448569 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:15.448527 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" podUID="bbc37446-f358-40b2-9dad-0041d189b76b" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.30:8443: connect: connection refused" Sep 09 06:51:15.448761 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:15.448625 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" podUID="bbc37446-f358-40b2-9dad-0041d189b76b" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": dial tcp 10.132.0.30:6567: connect: connection refused" Sep 09 06:51:15.765210 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:15.765120 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" event={"ID":"a0305aa7-15c7-4cb8-8d49-96e4437444a6","Type":"ContainerStarted","Data":"13affb8e13c20d2eba15d18c46c188f439652fb225afe821f90ea7d4c0465389"} Sep 09 06:51:20.790552 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:20.790523 2559 generic.go:356] "Generic (PLEG): container finished" podID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerID="13affb8e13c20d2eba15d18c46c188f439652fb225afe821f90ea7d4c0465389" exitCode=0 Sep 09 06:51:20.790906 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:20.790602 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" event={"ID":"a0305aa7-15c7-4cb8-8d49-96e4437444a6","Type":"ContainerDied","Data":"13affb8e13c20d2eba15d18c46c188f439652fb225afe821f90ea7d4c0465389"} Sep 09 06:51:21.801974 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:21.801926 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" event={"ID":"a0305aa7-15c7-4cb8-8d49-96e4437444a6","Type":"ContainerStarted","Data":"a26f6004d2eedabdde2d58edd6fc5b7d6bfc0d56da5f8cd146d1ae9a31143e2f"} Sep 09 06:51:21.802453 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:21.801990 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" event={"ID":"a0305aa7-15c7-4cb8-8d49-96e4437444a6","Type":"ContainerStarted","Data":"7b0537509c8551e842855534d3ff89cc22bd50a74ed0a6720c1e82767460090c"} Sep 09 06:51:21.802453 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:21.802011 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" event={"ID":"a0305aa7-15c7-4cb8-8d49-96e4437444a6","Type":"ContainerStarted","Data":"13c8e1848c6346ec154b75ad9c17423c99bb40fa11c97bdfbd4f9aa269242b2f"} Sep 09 06:51:21.802453 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:21.802031 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" event={"ID":"a0305aa7-15c7-4cb8-8d49-96e4437444a6","Type":"ContainerStarted","Data":"d9f5ff234d530d3f5c240b5436b02a5f53968d3022b142922a98f6c087fd2de4"} Sep 09 06:51:21.827476 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:21.827407 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" podStartSLOduration=13.827384279 podStartE2EDuration="13.827384279s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:51:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:51:21.824965576 +0000 UTC m=+779.721383024" watchObservedRunningTime="2026-09-09 06:51:21.827384279 +0000 UTC m=+779.723801702" Sep 09 06:51:24.218560 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:24.218509 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:51:24.219046 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:24.218577 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:51:24.219046 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:24.218598 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:51:24.219046 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:24.218616 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:51:24.221073 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:24.221011 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" podUID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.31:6571: connect: connection refused" Sep 09 06:51:24.221208 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:24.221063 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" podUID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerName="online" probeResult="failure" output="Get \"https://10.132.0.31:6567/health\": dial tcp 10.132.0.31:6567: connect: connection refused" Sep 09 06:51:24.221208 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:24.221077 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" podUID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.31:8816: connect: connection refused" Sep 09 06:51:24.221208 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:24.221151 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" podUID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.31:8443: connect: connection refused" Sep 09 06:51:25.448459 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:25.448410 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" podUID="bbc37446-f358-40b2-9dad-0041d189b76b" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": dial tcp 10.132.0.30:6567: connect: connection refused" Sep 09 06:51:25.448998 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:25.448433 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" podUID="bbc37446-f358-40b2-9dad-0041d189b76b" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.30:8443: connect: connection refused" Sep 09 06:51:27.219366 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:27.219311 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" podUID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.31:8443: connect: connection refused" Sep 09 06:51:27.219993 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:27.219958 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" podUID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.31:8816: connect: connection refused" Sep 09 06:51:27.220129 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:27.220015 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" podUID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.31:6571: connect: connection refused" Sep 09 06:51:27.220257 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:27.220134 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" podUID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerName="online" probeResult="failure" output="Get \"https://10.132.0.31:6567/health\": dial tcp 10.132.0.31:6567: connect: connection refused" Sep 09 06:51:29.219212 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:29.219165 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:51:29.219694 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:29.219233 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:51:29.219694 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:29.219253 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:51:29.219694 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:29.219269 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:51:30.219432 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:30.219394 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:51:30.219861 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:30.219552 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:51:30.219935 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:30.219911 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:51:30.220202 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:30.220183 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:51:30.220314 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:30.220255 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:51:30.224845 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:30.224825 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:51:30.860267 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:30.860235 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:51:30.863216 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:30.863193 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:51:35.448828 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:35.448777 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" podUID="bbc37446-f358-40b2-9dad-0041d189b76b" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.30:8443: connect: connection refused" Sep 09 06:51:35.449314 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:35.448784 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" podUID="bbc37446-f358-40b2-9dad-0041d189b76b" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": dial tcp 10.132.0.30:6567: connect: connection refused" Sep 09 06:51:35.449314 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:35.448928 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:51:35.449314 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:35.448991 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:51:38.572345 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.572321 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:51:38.582857 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.582832 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rx666\" (UniqueName: \"kubernetes.io/projected/bbc37446-f358-40b2-9dad-0041d189b76b-kube-api-access-rx666\") pod \"bbc37446-f358-40b2-9dad-0041d189b76b\" (UID: \"bbc37446-f358-40b2-9dad-0041d189b76b\") " Sep 09 06:51:38.582983 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.582870 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/bbc37446-f358-40b2-9dad-0041d189b76b-ui-tls\") pod \"bbc37446-f358-40b2-9dad-0041d189b76b\" (UID: \"bbc37446-f358-40b2-9dad-0041d189b76b\") " Sep 09 06:51:38.582983 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.582937 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bbc37446-f358-40b2-9dad-0041d189b76b-online-tls\") pod \"bbc37446-f358-40b2-9dad-0041d189b76b\" (UID: \"bbc37446-f358-40b2-9dad-0041d189b76b\") " Sep 09 06:51:38.582983 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.582962 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/bbc37446-f358-40b2-9dad-0041d189b76b-registry-tls\") pod \"bbc37446-f358-40b2-9dad-0041d189b76b\" (UID: \"bbc37446-f358-40b2-9dad-0041d189b76b\") " Sep 09 06:51:38.582983 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.582976 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/bbc37446-f358-40b2-9dad-0041d189b76b-offline-tls\") pod \"bbc37446-f358-40b2-9dad-0041d189b76b\" (UID: \"bbc37446-f358-40b2-9dad-0041d189b76b\") " Sep 09 06:51:38.583219 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.583019 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bbc37446-f358-40b2-9dad-0041d189b76b-feast-data\") pod \"bbc37446-f358-40b2-9dad-0041d189b76b\" (UID: \"bbc37446-f358-40b2-9dad-0041d189b76b\") " Sep 09 06:51:38.583799 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.583760 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbc37446-f358-40b2-9dad-0041d189b76b-feast-data" (OuterVolumeSpecName: "feast-data") pod "bbc37446-f358-40b2-9dad-0041d189b76b" (UID: "bbc37446-f358-40b2-9dad-0041d189b76b"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 06:51:38.585237 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.585207 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbc37446-f358-40b2-9dad-0041d189b76b-kube-api-access-rx666" (OuterVolumeSpecName: "kube-api-access-rx666") pod "bbc37446-f358-40b2-9dad-0041d189b76b" (UID: "bbc37446-f358-40b2-9dad-0041d189b76b"). InnerVolumeSpecName "kube-api-access-rx666". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 06:51:38.585482 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.585453 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbc37446-f358-40b2-9dad-0041d189b76b-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "bbc37446-f358-40b2-9dad-0041d189b76b" (UID: "bbc37446-f358-40b2-9dad-0041d189b76b"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 06:51:38.585593 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.585475 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbc37446-f358-40b2-9dad-0041d189b76b-online-tls" (OuterVolumeSpecName: "online-tls") pod "bbc37446-f358-40b2-9dad-0041d189b76b" (UID: "bbc37446-f358-40b2-9dad-0041d189b76b"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 06:51:38.585768 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.585749 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbc37446-f358-40b2-9dad-0041d189b76b-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "bbc37446-f358-40b2-9dad-0041d189b76b" (UID: "bbc37446-f358-40b2-9dad-0041d189b76b"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 06:51:38.585851 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.585777 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbc37446-f358-40b2-9dad-0041d189b76b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "bbc37446-f358-40b2-9dad-0041d189b76b" (UID: "bbc37446-f358-40b2-9dad-0041d189b76b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 06:51:38.684521 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.684495 2559 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/bbc37446-f358-40b2-9dad-0041d189b76b-ui-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:51:38.684654 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.684540 2559 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bbc37446-f358-40b2-9dad-0041d189b76b-online-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:51:38.684654 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.684551 2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/bbc37446-f358-40b2-9dad-0041d189b76b-registry-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:51:38.684654 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.684560 2559 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/bbc37446-f358-40b2-9dad-0041d189b76b-offline-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:51:38.684654 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.684568 2559 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bbc37446-f358-40b2-9dad-0041d189b76b-feast-data\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:51:38.684654 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.684576 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rx666\" (UniqueName: \"kubernetes.io/projected/bbc37446-f358-40b2-9dad-0041d189b76b-kube-api-access-rx666\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:51:38.899016 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.898985 2559 generic.go:356] "Generic (PLEG): container finished" podID="bbc37446-f358-40b2-9dad-0041d189b76b" containerID="64a63363668f558234bdc6e1ce5654bcdf4c245c5fc64de7938bd164fb19bb52" exitCode=137 Sep 09 06:51:38.899016 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.899011 2559 generic.go:356] "Generic (PLEG): container finished" podID="bbc37446-f358-40b2-9dad-0041d189b76b" containerID="03c8438c2669c502c613bfc2d12a2467eadfd500cfab76ecd9fc6f1d06187324" exitCode=137 Sep 09 06:51:38.899230 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.899047 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" Sep 09 06:51:38.899230 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.899079 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" event={"ID":"bbc37446-f358-40b2-9dad-0041d189b76b","Type":"ContainerDied","Data":"64a63363668f558234bdc6e1ce5654bcdf4c245c5fc64de7938bd164fb19bb52"} Sep 09 06:51:38.899230 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.899129 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" event={"ID":"bbc37446-f358-40b2-9dad-0041d189b76b","Type":"ContainerDied","Data":"03c8438c2669c502c613bfc2d12a2467eadfd500cfab76ecd9fc6f1d06187324"} Sep 09 06:51:38.899230 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.899149 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh" event={"ID":"bbc37446-f358-40b2-9dad-0041d189b76b","Type":"ContainerDied","Data":"21981ebca68c8e246aa076e236f178d7c66f240defa4cf888477e438743653e3"} Sep 09 06:51:38.899230 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.899169 2559 scope.go:117] "RemoveContainer" containerID="0c3d797079871daa8e9c4ce5c01ac989790a9579a4dcfb9776f46a09e5f5135b" Sep 09 06:51:38.907338 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.907321 2559 scope.go:117] "RemoveContainer" containerID="64a63363668f558234bdc6e1ce5654bcdf4c245c5fc64de7938bd164fb19bb52" Sep 09 06:51:38.914786 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.914764 2559 scope.go:117] "RemoveContainer" containerID="18edd35f88dee8daa0496921fc0b3a44c1856696ca5da28f5843c57f41b5f54f" Sep 09 06:51:38.917044 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.917024 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh"] Sep 09 06:51:38.922755 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.922731 2559 scope.go:117] "RemoveContainer" containerID="03c8438c2669c502c613bfc2d12a2467eadfd500cfab76ecd9fc6f1d06187324" Sep 09 06:51:38.923769 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.923750 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-84dccd7859-s6hgh"] Sep 09 06:51:38.930273 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.930041 2559 scope.go:117] "RemoveContainer" containerID="b099d479f742dddd0de7de5697605602104f8a489d34cb0219a92bc09e078193" Sep 09 06:51:38.937468 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.937447 2559 scope.go:117] "RemoveContainer" containerID="cf609aec6f863a485462cddb820140b0b0bdd86ce0c7c7d3749cfa25adbfdcfc" Sep 09 06:51:38.944461 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.944446 2559 scope.go:117] "RemoveContainer" containerID="0c3d797079871daa8e9c4ce5c01ac989790a9579a4dcfb9776f46a09e5f5135b" Sep 09 06:51:38.944737 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:51:38.944718 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c3d797079871daa8e9c4ce5c01ac989790a9579a4dcfb9776f46a09e5f5135b\": container with ID starting with 0c3d797079871daa8e9c4ce5c01ac989790a9579a4dcfb9776f46a09e5f5135b not found: ID does not exist" containerID="0c3d797079871daa8e9c4ce5c01ac989790a9579a4dcfb9776f46a09e5f5135b" Sep 09 06:51:38.944810 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.944747 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c3d797079871daa8e9c4ce5c01ac989790a9579a4dcfb9776f46a09e5f5135b"} err="failed to get container status \"0c3d797079871daa8e9c4ce5c01ac989790a9579a4dcfb9776f46a09e5f5135b\": rpc error: code = NotFound desc = could not find container \"0c3d797079871daa8e9c4ce5c01ac989790a9579a4dcfb9776f46a09e5f5135b\": container with ID starting with 0c3d797079871daa8e9c4ce5c01ac989790a9579a4dcfb9776f46a09e5f5135b not found: ID does not exist" Sep 09 06:51:38.944810 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.944764 2559 scope.go:117] "RemoveContainer" containerID="64a63363668f558234bdc6e1ce5654bcdf4c245c5fc64de7938bd164fb19bb52" Sep 09 06:51:38.945015 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:51:38.944996 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64a63363668f558234bdc6e1ce5654bcdf4c245c5fc64de7938bd164fb19bb52\": container with ID starting with 64a63363668f558234bdc6e1ce5654bcdf4c245c5fc64de7938bd164fb19bb52 not found: ID does not exist" containerID="64a63363668f558234bdc6e1ce5654bcdf4c245c5fc64de7938bd164fb19bb52" Sep 09 06:51:38.945065 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.945022 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64a63363668f558234bdc6e1ce5654bcdf4c245c5fc64de7938bd164fb19bb52"} err="failed to get container status \"64a63363668f558234bdc6e1ce5654bcdf4c245c5fc64de7938bd164fb19bb52\": rpc error: code = NotFound desc = could not find container \"64a63363668f558234bdc6e1ce5654bcdf4c245c5fc64de7938bd164fb19bb52\": container with ID starting with 64a63363668f558234bdc6e1ce5654bcdf4c245c5fc64de7938bd164fb19bb52 not found: ID does not exist" Sep 09 06:51:38.945065 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.945038 2559 scope.go:117] "RemoveContainer" containerID="18edd35f88dee8daa0496921fc0b3a44c1856696ca5da28f5843c57f41b5f54f" Sep 09 06:51:38.945304 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:51:38.945289 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18edd35f88dee8daa0496921fc0b3a44c1856696ca5da28f5843c57f41b5f54f\": container with ID starting with 18edd35f88dee8daa0496921fc0b3a44c1856696ca5da28f5843c57f41b5f54f not found: ID does not exist" containerID="18edd35f88dee8daa0496921fc0b3a44c1856696ca5da28f5843c57f41b5f54f" Sep 09 06:51:38.945348 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.945306 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18edd35f88dee8daa0496921fc0b3a44c1856696ca5da28f5843c57f41b5f54f"} err="failed to get container status \"18edd35f88dee8daa0496921fc0b3a44c1856696ca5da28f5843c57f41b5f54f\": rpc error: code = NotFound desc = could not find container \"18edd35f88dee8daa0496921fc0b3a44c1856696ca5da28f5843c57f41b5f54f\": container with ID starting with 18edd35f88dee8daa0496921fc0b3a44c1856696ca5da28f5843c57f41b5f54f not found: ID does not exist" Sep 09 06:51:38.945348 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.945320 2559 scope.go:117] "RemoveContainer" containerID="03c8438c2669c502c613bfc2d12a2467eadfd500cfab76ecd9fc6f1d06187324" Sep 09 06:51:38.945523 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:51:38.945508 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03c8438c2669c502c613bfc2d12a2467eadfd500cfab76ecd9fc6f1d06187324\": container with ID starting with 03c8438c2669c502c613bfc2d12a2467eadfd500cfab76ecd9fc6f1d06187324 not found: ID does not exist" containerID="03c8438c2669c502c613bfc2d12a2467eadfd500cfab76ecd9fc6f1d06187324" Sep 09 06:51:38.945562 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.945529 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03c8438c2669c502c613bfc2d12a2467eadfd500cfab76ecd9fc6f1d06187324"} err="failed to get container status \"03c8438c2669c502c613bfc2d12a2467eadfd500cfab76ecd9fc6f1d06187324\": rpc error: code = NotFound desc = could not find container \"03c8438c2669c502c613bfc2d12a2467eadfd500cfab76ecd9fc6f1d06187324\": container with ID starting with 03c8438c2669c502c613bfc2d12a2467eadfd500cfab76ecd9fc6f1d06187324 not found: ID does not exist" Sep 09 06:51:38.945562 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.945542 2559 scope.go:117] "RemoveContainer" containerID="b099d479f742dddd0de7de5697605602104f8a489d34cb0219a92bc09e078193" Sep 09 06:51:38.945737 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:51:38.945723 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b099d479f742dddd0de7de5697605602104f8a489d34cb0219a92bc09e078193\": container with ID starting with b099d479f742dddd0de7de5697605602104f8a489d34cb0219a92bc09e078193 not found: ID does not exist" containerID="b099d479f742dddd0de7de5697605602104f8a489d34cb0219a92bc09e078193" Sep 09 06:51:38.945773 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.945739 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b099d479f742dddd0de7de5697605602104f8a489d34cb0219a92bc09e078193"} err="failed to get container status \"b099d479f742dddd0de7de5697605602104f8a489d34cb0219a92bc09e078193\": rpc error: code = NotFound desc = could not find container \"b099d479f742dddd0de7de5697605602104f8a489d34cb0219a92bc09e078193\": container with ID starting with b099d479f742dddd0de7de5697605602104f8a489d34cb0219a92bc09e078193 not found: ID does not exist" Sep 09 06:51:38.945773 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.945755 2559 scope.go:117] "RemoveContainer" containerID="cf609aec6f863a485462cddb820140b0b0bdd86ce0c7c7d3749cfa25adbfdcfc" Sep 09 06:51:38.945975 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:51:38.945959 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf609aec6f863a485462cddb820140b0b0bdd86ce0c7c7d3749cfa25adbfdcfc\": container with ID starting with cf609aec6f863a485462cddb820140b0b0bdd86ce0c7c7d3749cfa25adbfdcfc not found: ID does not exist" containerID="cf609aec6f863a485462cddb820140b0b0bdd86ce0c7c7d3749cfa25adbfdcfc" Sep 09 06:51:38.946022 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.945979 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf609aec6f863a485462cddb820140b0b0bdd86ce0c7c7d3749cfa25adbfdcfc"} err="failed to get container status \"cf609aec6f863a485462cddb820140b0b0bdd86ce0c7c7d3749cfa25adbfdcfc\": rpc error: code = NotFound desc = could not find container \"cf609aec6f863a485462cddb820140b0b0bdd86ce0c7c7d3749cfa25adbfdcfc\": container with ID starting with cf609aec6f863a485462cddb820140b0b0bdd86ce0c7c7d3749cfa25adbfdcfc not found: ID does not exist" Sep 09 06:51:38.946022 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.945993 2559 scope.go:117] "RemoveContainer" containerID="0c3d797079871daa8e9c4ce5c01ac989790a9579a4dcfb9776f46a09e5f5135b" Sep 09 06:51:38.946239 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.946222 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c3d797079871daa8e9c4ce5c01ac989790a9579a4dcfb9776f46a09e5f5135b"} err="failed to get container status \"0c3d797079871daa8e9c4ce5c01ac989790a9579a4dcfb9776f46a09e5f5135b\": rpc error: code = NotFound desc = could not find container \"0c3d797079871daa8e9c4ce5c01ac989790a9579a4dcfb9776f46a09e5f5135b\": container with ID starting with 0c3d797079871daa8e9c4ce5c01ac989790a9579a4dcfb9776f46a09e5f5135b not found: ID does not exist" Sep 09 06:51:38.946300 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.946240 2559 scope.go:117] "RemoveContainer" containerID="64a63363668f558234bdc6e1ce5654bcdf4c245c5fc64de7938bd164fb19bb52" Sep 09 06:51:38.946470 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.946453 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64a63363668f558234bdc6e1ce5654bcdf4c245c5fc64de7938bd164fb19bb52"} err="failed to get container status \"64a63363668f558234bdc6e1ce5654bcdf4c245c5fc64de7938bd164fb19bb52\": rpc error: code = NotFound desc = could not find container \"64a63363668f558234bdc6e1ce5654bcdf4c245c5fc64de7938bd164fb19bb52\": container with ID starting with 64a63363668f558234bdc6e1ce5654bcdf4c245c5fc64de7938bd164fb19bb52 not found: ID does not exist" Sep 09 06:51:38.946519 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.946470 2559 scope.go:117] "RemoveContainer" containerID="18edd35f88dee8daa0496921fc0b3a44c1856696ca5da28f5843c57f41b5f54f" Sep 09 06:51:38.946715 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.946697 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18edd35f88dee8daa0496921fc0b3a44c1856696ca5da28f5843c57f41b5f54f"} err="failed to get container status \"18edd35f88dee8daa0496921fc0b3a44c1856696ca5da28f5843c57f41b5f54f\": rpc error: code = NotFound desc = could not find container \"18edd35f88dee8daa0496921fc0b3a44c1856696ca5da28f5843c57f41b5f54f\": container with ID starting with 18edd35f88dee8daa0496921fc0b3a44c1856696ca5da28f5843c57f41b5f54f not found: ID does not exist" Sep 09 06:51:38.946788 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.946715 2559 scope.go:117] "RemoveContainer" containerID="03c8438c2669c502c613bfc2d12a2467eadfd500cfab76ecd9fc6f1d06187324" Sep 09 06:51:38.946937 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.946915 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03c8438c2669c502c613bfc2d12a2467eadfd500cfab76ecd9fc6f1d06187324"} err="failed to get container status \"03c8438c2669c502c613bfc2d12a2467eadfd500cfab76ecd9fc6f1d06187324\": rpc error: code = NotFound desc = could not find container \"03c8438c2669c502c613bfc2d12a2467eadfd500cfab76ecd9fc6f1d06187324\": container with ID starting with 03c8438c2669c502c613bfc2d12a2467eadfd500cfab76ecd9fc6f1d06187324 not found: ID does not exist" Sep 09 06:51:38.946937 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.946936 2559 scope.go:117] "RemoveContainer" containerID="b099d479f742dddd0de7de5697605602104f8a489d34cb0219a92bc09e078193" Sep 09 06:51:38.947246 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.947229 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b099d479f742dddd0de7de5697605602104f8a489d34cb0219a92bc09e078193"} err="failed to get container status \"b099d479f742dddd0de7de5697605602104f8a489d34cb0219a92bc09e078193\": rpc error: code = NotFound desc = could not find container \"b099d479f742dddd0de7de5697605602104f8a489d34cb0219a92bc09e078193\": container with ID starting with b099d479f742dddd0de7de5697605602104f8a489d34cb0219a92bc09e078193 not found: ID does not exist" Sep 09 06:51:38.947292 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.947247 2559 scope.go:117] "RemoveContainer" containerID="cf609aec6f863a485462cddb820140b0b0bdd86ce0c7c7d3749cfa25adbfdcfc" Sep 09 06:51:38.947460 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:38.947445 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf609aec6f863a485462cddb820140b0b0bdd86ce0c7c7d3749cfa25adbfdcfc"} err="failed to get container status \"cf609aec6f863a485462cddb820140b0b0bdd86ce0c7c7d3749cfa25adbfdcfc\": rpc error: code = NotFound desc = could not find container \"cf609aec6f863a485462cddb820140b0b0bdd86ce0c7c7d3749cfa25adbfdcfc\": container with ID starting with cf609aec6f863a485462cddb820140b0b0bdd86ce0c7c7d3749cfa25adbfdcfc not found: ID does not exist" Sep 09 06:51:40.702810 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:51:40.702776 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbc37446-f358-40b2-9dad-0041d189b76b" path="/var/lib/kubelet/pods/bbc37446-f358-40b2-9dad-0041d189b76b/volumes" Sep 09 06:52:33.313524 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.313487 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b"] Sep 09 06:52:33.313918 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.313846 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbc37446-f358-40b2-9dad-0041d189b76b" containerName="offline" Sep 09 06:52:33.313918 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.313856 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbc37446-f358-40b2-9dad-0041d189b76b" containerName="offline" Sep 09 06:52:33.313918 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.313870 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbc37446-f358-40b2-9dad-0041d189b76b" containerName="registry" Sep 09 06:52:33.313918 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.313876 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbc37446-f358-40b2-9dad-0041d189b76b" containerName="registry" Sep 09 06:52:33.313918 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.313885 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbc37446-f358-40b2-9dad-0041d189b76b" containerName="feast-apply" Sep 09 06:52:33.313918 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.313890 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbc37446-f358-40b2-9dad-0041d189b76b" containerName="feast-apply" Sep 09 06:52:33.313918 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.313905 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbc37446-f358-40b2-9dad-0041d189b76b" containerName="online" Sep 09 06:52:33.313918 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.313910 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbc37446-f358-40b2-9dad-0041d189b76b" containerName="online" Sep 09 06:52:33.313918 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.313920 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbc37446-f358-40b2-9dad-0041d189b76b" containerName="ui" Sep 09 06:52:33.314207 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.313925 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbc37446-f358-40b2-9dad-0041d189b76b" containerName="ui" Sep 09 06:52:33.314207 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.313935 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbc37446-f358-40b2-9dad-0041d189b76b" containerName="feast-init" Sep 09 06:52:33.314207 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.313940 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbc37446-f358-40b2-9dad-0041d189b76b" containerName="feast-init" Sep 09 06:52:33.314207 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.313990 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbc37446-f358-40b2-9dad-0041d189b76b" containerName="ui" Sep 09 06:52:33.314207 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.313997 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbc37446-f358-40b2-9dad-0041d189b76b" containerName="offline" Sep 09 06:52:33.314207 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.314003 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbc37446-f358-40b2-9dad-0041d189b76b" containerName="registry" Sep 09 06:52:33.314207 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.314008 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbc37446-f358-40b2-9dad-0041d189b76b" containerName="online" Sep 09 06:52:33.318761 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.318741 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:52:33.321098 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.321075 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 09 06:52:33.321098 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.321091 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-pp2b2\"" Sep 09 06:52:33.321287 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.321076 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 09 06:52:33.321412 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.321390 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\"" Sep 09 06:52:33.321478 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.321400 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 09 06:52:33.321612 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.321600 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 09 06:52:33.321685 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.321656 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 09 06:52:33.321798 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.321782 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 09 06:52:33.329545 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.329523 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b"] Sep 09 06:52:33.332852 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.332831 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87bdr\" (UniqueName: \"kubernetes.io/projected/4b0421c4-6824-412c-aef0-8befad6975bb-kube-api-access-87bdr\") pod \"feast-simple-feast-remote-setup-59f4c548f5-8tm6b\" (UID: \"4b0421c4-6824-412c-aef0-8befad6975bb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:52:33.332938 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.332883 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4b0421c4-6824-412c-aef0-8befad6975bb-ui-tls\") pod \"feast-simple-feast-remote-setup-59f4c548f5-8tm6b\" (UID: \"4b0421c4-6824-412c-aef0-8befad6975bb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:52:33.332938 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.332904 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4b0421c4-6824-412c-aef0-8befad6975bb-online-tls\") pod \"feast-simple-feast-remote-setup-59f4c548f5-8tm6b\" (UID: \"4b0421c4-6824-412c-aef0-8befad6975bb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:52:33.333037 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.332961 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4b0421c4-6824-412c-aef0-8befad6975bb-offline-tls\") pod \"feast-simple-feast-remote-setup-59f4c548f5-8tm6b\" (UID: \"4b0421c4-6824-412c-aef0-8befad6975bb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:52:33.333111 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.333047 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/4b0421c4-6824-412c-aef0-8befad6975bb-registry-tls\") pod \"feast-simple-feast-remote-setup-59f4c548f5-8tm6b\" (UID: \"4b0421c4-6824-412c-aef0-8befad6975bb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:52:33.333111 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.333100 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4b0421c4-6824-412c-aef0-8befad6975bb-feast-data\") pod \"feast-simple-feast-remote-setup-59f4c548f5-8tm6b\" (UID: \"4b0421c4-6824-412c-aef0-8befad6975bb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:52:33.434113 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.434077 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4b0421c4-6824-412c-aef0-8befad6975bb-ui-tls\") pod \"feast-simple-feast-remote-setup-59f4c548f5-8tm6b\" (UID: \"4b0421c4-6824-412c-aef0-8befad6975bb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:52:33.434256 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.434133 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4b0421c4-6824-412c-aef0-8befad6975bb-online-tls\") pod \"feast-simple-feast-remote-setup-59f4c548f5-8tm6b\" (UID: \"4b0421c4-6824-412c-aef0-8befad6975bb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:52:33.434256 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:52:33.434151 2559 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 09 06:52:33.434256 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.434171 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4b0421c4-6824-412c-aef0-8befad6975bb-offline-tls\") pod \"feast-simple-feast-remote-setup-59f4c548f5-8tm6b\" (UID: \"4b0421c4-6824-412c-aef0-8befad6975bb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:52:33.434256 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:52:33.434220 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b0421c4-6824-412c-aef0-8befad6975bb-ui-tls podName:4b0421c4-6824-412c-aef0-8befad6975bb nodeName:}" failed. No retries permitted until 2026-09-09 06:52:33.934200265 +0000 UTC m=+851.830617682 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/4b0421c4-6824-412c-aef0-8befad6975bb-ui-tls") pod "feast-simple-feast-remote-setup-59f4c548f5-8tm6b" (UID: "4b0421c4-6824-412c-aef0-8befad6975bb") : secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 09 06:52:33.434256 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:52:33.434251 2559 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-offline-tls: secret "feast-simple-feast-remote-setup-offline-tls" not found Sep 09 06:52:33.434256 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.434252 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/4b0421c4-6824-412c-aef0-8befad6975bb-registry-tls\") pod \"feast-simple-feast-remote-setup-59f4c548f5-8tm6b\" (UID: \"4b0421c4-6824-412c-aef0-8befad6975bb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:52:33.434531 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.434277 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4b0421c4-6824-412c-aef0-8befad6975bb-feast-data\") pod \"feast-simple-feast-remote-setup-59f4c548f5-8tm6b\" (UID: \"4b0421c4-6824-412c-aef0-8befad6975bb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:52:33.434531 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:52:33.434315 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b0421c4-6824-412c-aef0-8befad6975bb-offline-tls podName:4b0421c4-6824-412c-aef0-8befad6975bb nodeName:}" failed. No retries permitted until 2026-09-09 06:52:33.934303515 +0000 UTC m=+851.830720915 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/4b0421c4-6824-412c-aef0-8befad6975bb-offline-tls") pod "feast-simple-feast-remote-setup-59f4c548f5-8tm6b" (UID: "4b0421c4-6824-412c-aef0-8befad6975bb") : secret "feast-simple-feast-remote-setup-offline-tls" not found Sep 09 06:52:33.434531 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:52:33.434252 2559 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-online-tls: secret "feast-simple-feast-remote-setup-online-tls" not found Sep 09 06:52:33.434531 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.434356 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87bdr\" (UniqueName: \"kubernetes.io/projected/4b0421c4-6824-412c-aef0-8befad6975bb-kube-api-access-87bdr\") pod \"feast-simple-feast-remote-setup-59f4c548f5-8tm6b\" (UID: \"4b0421c4-6824-412c-aef0-8befad6975bb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:52:33.434531 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:52:33.434409 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b0421c4-6824-412c-aef0-8befad6975bb-online-tls podName:4b0421c4-6824-412c-aef0-8befad6975bb nodeName:}" failed. No retries permitted until 2026-09-09 06:52:33.934389394 +0000 UTC m=+851.830806813 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/4b0421c4-6824-412c-aef0-8befad6975bb-online-tls") pod "feast-simple-feast-remote-setup-59f4c548f5-8tm6b" (UID: "4b0421c4-6824-412c-aef0-8befad6975bb") : secret "feast-simple-feast-remote-setup-online-tls" not found Sep 09 06:52:33.434695 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.434657 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4b0421c4-6824-412c-aef0-8befad6975bb-feast-data\") pod \"feast-simple-feast-remote-setup-59f4c548f5-8tm6b\" (UID: \"4b0421c4-6824-412c-aef0-8befad6975bb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:52:33.434918 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.434899 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/4b0421c4-6824-412c-aef0-8befad6975bb-registry-tls\") pod \"feast-simple-feast-remote-setup-59f4c548f5-8tm6b\" (UID: \"4b0421c4-6824-412c-aef0-8befad6975bb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:52:33.442656 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.442628 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87bdr\" (UniqueName: \"kubernetes.io/projected/4b0421c4-6824-412c-aef0-8befad6975bb-kube-api-access-87bdr\") pod \"feast-simple-feast-remote-setup-59f4c548f5-8tm6b\" (UID: \"4b0421c4-6824-412c-aef0-8befad6975bb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:52:33.938611 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.938573 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4b0421c4-6824-412c-aef0-8befad6975bb-ui-tls\") pod \"feast-simple-feast-remote-setup-59f4c548f5-8tm6b\" (UID: \"4b0421c4-6824-412c-aef0-8befad6975bb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:52:33.938782 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.938619 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4b0421c4-6824-412c-aef0-8befad6975bb-online-tls\") pod \"feast-simple-feast-remote-setup-59f4c548f5-8tm6b\" (UID: \"4b0421c4-6824-412c-aef0-8befad6975bb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:52:33.938782 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.938698 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4b0421c4-6824-412c-aef0-8befad6975bb-offline-tls\") pod \"feast-simple-feast-remote-setup-59f4c548f5-8tm6b\" (UID: \"4b0421c4-6824-412c-aef0-8befad6975bb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:52:33.941047 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.941019 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4b0421c4-6824-412c-aef0-8befad6975bb-ui-tls\") pod \"feast-simple-feast-remote-setup-59f4c548f5-8tm6b\" (UID: \"4b0421c4-6824-412c-aef0-8befad6975bb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:52:33.941178 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.941109 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4b0421c4-6824-412c-aef0-8befad6975bb-online-tls\") pod \"feast-simple-feast-remote-setup-59f4c548f5-8tm6b\" (UID: \"4b0421c4-6824-412c-aef0-8befad6975bb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:52:33.941249 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:33.941228 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4b0421c4-6824-412c-aef0-8befad6975bb-offline-tls\") pod \"feast-simple-feast-remote-setup-59f4c548f5-8tm6b\" (UID: \"4b0421c4-6824-412c-aef0-8befad6975bb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:52:34.228265 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:34.228175 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:52:34.506339 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:34.506315 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b"] Sep 09 06:52:34.508263 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:52:34.508234 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b0421c4_6824_412c_aef0_8befad6975bb.slice/crio-812654ea6ce257b189ceed08f23d6bfb70a6311135fd97c6fb81db960253c9ab WatchSource:0}: Error finding container 812654ea6ce257b189ceed08f23d6bfb70a6311135fd97c6fb81db960253c9ab: Status 404 returned error can't find the container with id 812654ea6ce257b189ceed08f23d6bfb70a6311135fd97c6fb81db960253c9ab Sep 09 06:52:35.158376 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:35.158338 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" event={"ID":"4b0421c4-6824-412c-aef0-8befad6975bb","Type":"ContainerStarted","Data":"9ecd681b054116ac1726014b9212d2da21d2cea86e77b310b12e2883b8564aaf"} Sep 09 06:52:35.158552 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:35.158384 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" event={"ID":"4b0421c4-6824-412c-aef0-8befad6975bb","Type":"ContainerStarted","Data":"812654ea6ce257b189ceed08f23d6bfb70a6311135fd97c6fb81db960253c9ab"} Sep 09 06:52:40.183888 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:40.183852 2559 generic.go:356] "Generic (PLEG): container finished" podID="4b0421c4-6824-412c-aef0-8befad6975bb" containerID="9ecd681b054116ac1726014b9212d2da21d2cea86e77b310b12e2883b8564aaf" exitCode=0 Sep 09 06:52:40.184307 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:40.183929 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" event={"ID":"4b0421c4-6824-412c-aef0-8befad6975bb","Type":"ContainerDied","Data":"9ecd681b054116ac1726014b9212d2da21d2cea86e77b310b12e2883b8564aaf"} Sep 09 06:52:41.190949 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:41.190916 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" event={"ID":"4b0421c4-6824-412c-aef0-8befad6975bb","Type":"ContainerStarted","Data":"8ee3cc1da241178c6b3ea8ca2852c8b2f65f352022c1776bd2f27a7f753d13ba"} Sep 09 06:52:59.287113 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:59.287081 2559 generic.go:356] "Generic (PLEG): container finished" podID="4b0421c4-6824-412c-aef0-8befad6975bb" containerID="8ee3cc1da241178c6b3ea8ca2852c8b2f65f352022c1776bd2f27a7f753d13ba" exitCode=0 Sep 09 06:52:59.287490 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:52:59.287164 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" event={"ID":"4b0421c4-6824-412c-aef0-8befad6975bb","Type":"ContainerDied","Data":"8ee3cc1da241178c6b3ea8ca2852c8b2f65f352022c1776bd2f27a7f753d13ba"} Sep 09 06:53:00.297443 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:53:00.297396 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" event={"ID":"4b0421c4-6824-412c-aef0-8befad6975bb","Type":"ContainerStarted","Data":"749b5a0081d4cd295292a2bbe3d8eeb6cdf2c4bcff4ec98ca96586602e4d74f2"} Sep 09 06:53:00.297962 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:53:00.297465 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" event={"ID":"4b0421c4-6824-412c-aef0-8befad6975bb","Type":"ContainerStarted","Data":"ab5d1ed754e47bd4a353c91a2076e8aa09cf4114d1a68acb630dd24b27961f2b"} Sep 09 06:53:00.297962 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:53:00.297487 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" event={"ID":"4b0421c4-6824-412c-aef0-8befad6975bb","Type":"ContainerStarted","Data":"f13c9bba2972d6fe62603fdda086227f35da31041f3a3d7fbd6ab2e0320d7410"} Sep 09 06:53:00.325568 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:53:00.325507 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" podStartSLOduration=27.325489525 podStartE2EDuration="27.325489525s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:52:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:53:00.323142931 +0000 UTC m=+878.219560353" watchObservedRunningTime="2026-09-09 06:53:00.325489525 +0000 UTC m=+878.221906947" Sep 09 06:53:01.228951 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:53:01.228908 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:53:01.229178 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:53:01.228975 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:53:01.229178 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:53:01.228992 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:53:01.230382 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:53:01.230342 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" podUID="4b0421c4-6824-412c-aef0-8befad6975bb" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.32:8816: connect: connection refused" Sep 09 06:53:01.230382 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:53:01.230368 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" podUID="4b0421c4-6824-412c-aef0-8befad6975bb" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.32:8443: connect: connection refused" Sep 09 06:53:01.230560 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:53:01.230386 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" podUID="4b0421c4-6824-412c-aef0-8befad6975bb" containerName="online" probeResult="failure" output="Get \"https://10.132.0.32:6567/health\": dial tcp 10.132.0.32:6567: connect: connection refused" Sep 09 06:53:04.228627 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:53:04.228584 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:53:04.229216 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:53:04.229190 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:53:04.229216 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:53:04.228836 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" podUID="4b0421c4-6824-412c-aef0-8befad6975bb" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.32:8443: connect: connection refused" Sep 09 06:53:04.229405 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:53:04.229225 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:53:04.229405 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:53:04.228845 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" podUID="4b0421c4-6824-412c-aef0-8befad6975bb" containerName="online" probeResult="failure" output="Get \"https://10.132.0.32:6567/health\": dial tcp 10.132.0.32:6567: connect: connection refused" Sep 09 06:53:04.229405 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:53:04.228856 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" podUID="4b0421c4-6824-412c-aef0-8befad6975bb" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.32:8816: connect: connection refused" Sep 09 06:53:07.229338 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:53:07.229288 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" podUID="4b0421c4-6824-412c-aef0-8befad6975bb" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.32:8443: connect: connection refused" Sep 09 06:53:07.230118 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:53:07.230090 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:53:07.231013 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:53:07.230992 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:53:07.248882 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:53:07.248857 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:53:07.354218 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:53:07.354189 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:53:10.229391 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:53:10.229355 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:53:10.352246 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:53:10.352217 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:53:22.529628 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:53:22.529559 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nf9gf_26d28788-4f98-454b-ab39-2d0d857aa674/console-operator/2.log" Sep 09 06:53:22.531844 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:53:22.531821 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nf9gf_26d28788-4f98-454b-ab39-2d0d857aa674/console-operator/2.log" Sep 09 06:54:11.707261 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:11.707224 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b"] Sep 09 06:54:11.707763 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:11.707519 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" podUID="4b0421c4-6824-412c-aef0-8befad6975bb" containerName="online" containerID="cri-o://f13c9bba2972d6fe62603fdda086227f35da31041f3a3d7fbd6ab2e0320d7410" gracePeriod=30 Sep 09 06:54:11.707763 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:11.707620 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" podUID="4b0421c4-6824-412c-aef0-8befad6975bb" containerName="ui" containerID="cri-o://749b5a0081d4cd295292a2bbe3d8eeb6cdf2c4bcff4ec98ca96586602e4d74f2" gracePeriod=30 Sep 09 06:54:11.707763 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:11.707695 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" podUID="4b0421c4-6824-412c-aef0-8befad6975bb" containerName="offline" containerID="cri-o://ab5d1ed754e47bd4a353c91a2076e8aa09cf4114d1a68acb630dd24b27961f2b" gracePeriod=30 Sep 09 06:54:11.804031 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:11.803995 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp"] Sep 09 06:54:11.804314 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:11.804257 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" podUID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerName="registry" containerID="cri-o://d9f5ff234d530d3f5c240b5436b02a5f53968d3022b142922a98f6c087fd2de4" gracePeriod=30 Sep 09 06:54:11.804400 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:11.804309 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" podUID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerName="online" containerID="cri-o://13c8e1848c6346ec154b75ad9c17423c99bb40fa11c97bdfbd4f9aa269242b2f" gracePeriod=30 Sep 09 06:54:11.804400 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:11.804310 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" podUID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerName="offline" containerID="cri-o://7b0537509c8551e842855534d3ff89cc22bd50a74ed0a6720c1e82767460090c" gracePeriod=30 Sep 09 06:54:11.804508 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:11.804324 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" podUID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerName="ui" containerID="cri-o://a26f6004d2eedabdde2d58edd6fc5b7d6bfc0d56da5f8cd146d1ae9a31143e2f" gracePeriod=30 Sep 09 06:54:12.646480 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:12.646413 2559 generic.go:356] "Generic (PLEG): container finished" podID="4b0421c4-6824-412c-aef0-8befad6975bb" containerID="749b5a0081d4cd295292a2bbe3d8eeb6cdf2c4bcff4ec98ca96586602e4d74f2" exitCode=0 Sep 09 06:54:12.646480 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:12.646438 2559 generic.go:356] "Generic (PLEG): container finished" podID="4b0421c4-6824-412c-aef0-8befad6975bb" containerID="f13c9bba2972d6fe62603fdda086227f35da31041f3a3d7fbd6ab2e0320d7410" exitCode=0 Sep 09 06:54:12.646688 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:12.646489 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" event={"ID":"4b0421c4-6824-412c-aef0-8befad6975bb","Type":"ContainerDied","Data":"749b5a0081d4cd295292a2bbe3d8eeb6cdf2c4bcff4ec98ca96586602e4d74f2"} Sep 09 06:54:12.646688 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:12.646534 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" event={"ID":"4b0421c4-6824-412c-aef0-8befad6975bb","Type":"ContainerDied","Data":"f13c9bba2972d6fe62603fdda086227f35da31041f3a3d7fbd6ab2e0320d7410"} Sep 09 06:54:12.649557 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:12.649536 2559 generic.go:356] "Generic (PLEG): container finished" podID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerID="a26f6004d2eedabdde2d58edd6fc5b7d6bfc0d56da5f8cd146d1ae9a31143e2f" exitCode=0 Sep 09 06:54:12.649557 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:12.649557 2559 generic.go:356] "Generic (PLEG): container finished" podID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerID="13c8e1848c6346ec154b75ad9c17423c99bb40fa11c97bdfbd4f9aa269242b2f" exitCode=0 Sep 09 06:54:12.649687 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:12.649606 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" event={"ID":"a0305aa7-15c7-4cb8-8d49-96e4437444a6","Type":"ContainerDied","Data":"a26f6004d2eedabdde2d58edd6fc5b7d6bfc0d56da5f8cd146d1ae9a31143e2f"} Sep 09 06:54:12.649687 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:12.649639 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" event={"ID":"a0305aa7-15c7-4cb8-8d49-96e4437444a6","Type":"ContainerDied","Data":"13c8e1848c6346ec154b75ad9c17423c99bb40fa11c97bdfbd4f9aa269242b2f"} Sep 09 06:54:17.337338 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:17.337291 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" podUID="4b0421c4-6824-412c-aef0-8befad6975bb" containerName="online" probeResult="failure" output="Get \"https://10.132.0.32:6567/health\": dial tcp 10.132.0.32:6567: connect: connection refused" Sep 09 06:54:20.220596 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:20.220545 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" podUID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.31:8443: connect: connection refused" Sep 09 06:54:20.351851 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:20.351813 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" podUID="4b0421c4-6824-412c-aef0-8befad6975bb" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.32:8443: connect: connection refused" Sep 09 06:54:20.860150 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:20.860102 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" podUID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerName="online" probeResult="failure" output="Get \"https://10.132.0.31:6567/health\": dial tcp 10.132.0.31:6567: connect: connection refused" Sep 09 06:54:27.337355 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:27.337308 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" podUID="4b0421c4-6824-412c-aef0-8befad6975bb" containerName="online" probeResult="failure" output="Get \"https://10.132.0.32:6567/health\": dial tcp 10.132.0.32:6567: connect: connection refused" Sep 09 06:54:30.221038 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:30.220980 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" podUID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.31:8443: connect: connection refused" Sep 09 06:54:30.352375 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:30.352333 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" podUID="4b0421c4-6824-412c-aef0-8befad6975bb" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.32:8443: connect: connection refused" Sep 09 06:54:30.860251 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:30.860207 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" podUID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerName="online" probeResult="failure" output="Get \"https://10.132.0.31:6567/health\": dial tcp 10.132.0.31:6567: connect: connection refused" Sep 09 06:54:37.337008 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:37.336969 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" podUID="4b0421c4-6824-412c-aef0-8befad6975bb" containerName="online" probeResult="failure" output="Get \"https://10.132.0.32:6567/health\": dial tcp 10.132.0.32:6567: connect: connection refused" Sep 09 06:54:37.337429 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:37.337091 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:54:40.220740 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:40.220683 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" podUID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.31:8443: connect: connection refused" Sep 09 06:54:40.221237 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:40.220854 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:54:40.352542 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:40.352501 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" podUID="4b0421c4-6824-412c-aef0-8befad6975bb" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.32:8443: connect: connection refused" Sep 09 06:54:40.352706 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:40.352629 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:54:40.860910 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:40.860865 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" podUID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerName="online" probeResult="failure" output="Get \"https://10.132.0.31:6567/health\": dial tcp 10.132.0.31:6567: connect: connection refused" Sep 09 06:54:40.861106 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:40.860987 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:54:41.793559 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:41.793527 2559 generic.go:356] "Generic (PLEG): container finished" podID="4b0421c4-6824-412c-aef0-8befad6975bb" containerID="ab5d1ed754e47bd4a353c91a2076e8aa09cf4114d1a68acb630dd24b27961f2b" exitCode=137 Sep 09 06:54:41.793874 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:41.793576 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" event={"ID":"4b0421c4-6824-412c-aef0-8befad6975bb","Type":"ContainerDied","Data":"ab5d1ed754e47bd4a353c91a2076e8aa09cf4114d1a68acb630dd24b27961f2b"} Sep 09 06:54:42.354739 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.354709 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:54:42.391904 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.391881 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4b0421c4-6824-412c-aef0-8befad6975bb-online-tls\") pod \"4b0421c4-6824-412c-aef0-8befad6975bb\" (UID: \"4b0421c4-6824-412c-aef0-8befad6975bb\") " Sep 09 06:54:42.392022 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.391920 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4b0421c4-6824-412c-aef0-8befad6975bb-offline-tls\") pod \"4b0421c4-6824-412c-aef0-8befad6975bb\" (UID: \"4b0421c4-6824-412c-aef0-8befad6975bb\") " Sep 09 06:54:42.392022 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.391947 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-87bdr\" (UniqueName: \"kubernetes.io/projected/4b0421c4-6824-412c-aef0-8befad6975bb-kube-api-access-87bdr\") pod \"4b0421c4-6824-412c-aef0-8befad6975bb\" (UID: \"4b0421c4-6824-412c-aef0-8befad6975bb\") " Sep 09 06:54:42.392022 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.391982 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4b0421c4-6824-412c-aef0-8befad6975bb-ui-tls\") pod \"4b0421c4-6824-412c-aef0-8befad6975bb\" (UID: \"4b0421c4-6824-412c-aef0-8befad6975bb\") " Sep 09 06:54:42.392022 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.392006 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/4b0421c4-6824-412c-aef0-8befad6975bb-registry-tls\") pod \"4b0421c4-6824-412c-aef0-8befad6975bb\" (UID: \"4b0421c4-6824-412c-aef0-8befad6975bb\") " Sep 09 06:54:42.392391 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.392030 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4b0421c4-6824-412c-aef0-8befad6975bb-feast-data\") pod \"4b0421c4-6824-412c-aef0-8befad6975bb\" (UID: \"4b0421c4-6824-412c-aef0-8befad6975bb\") " Sep 09 06:54:42.392514 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.392461 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b0421c4-6824-412c-aef0-8befad6975bb-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4b0421c4-6824-412c-aef0-8befad6975bb" (UID: "4b0421c4-6824-412c-aef0-8befad6975bb"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 06:54:42.392749 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.392723 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b0421c4-6824-412c-aef0-8befad6975bb-feast-data" (OuterVolumeSpecName: "feast-data") pod "4b0421c4-6824-412c-aef0-8befad6975bb" (UID: "4b0421c4-6824-412c-aef0-8befad6975bb"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 06:54:42.394335 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.394306 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b0421c4-6824-412c-aef0-8befad6975bb-kube-api-access-87bdr" (OuterVolumeSpecName: "kube-api-access-87bdr") pod "4b0421c4-6824-412c-aef0-8befad6975bb" (UID: "4b0421c4-6824-412c-aef0-8befad6975bb"). InnerVolumeSpecName "kube-api-access-87bdr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 06:54:42.394631 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.394601 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b0421c4-6824-412c-aef0-8befad6975bb-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "4b0421c4-6824-412c-aef0-8befad6975bb" (UID: "4b0421c4-6824-412c-aef0-8befad6975bb"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 06:54:42.394700 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.394622 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b0421c4-6824-412c-aef0-8befad6975bb-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "4b0421c4-6824-412c-aef0-8befad6975bb" (UID: "4b0421c4-6824-412c-aef0-8befad6975bb"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 06:54:42.394813 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.394790 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b0421c4-6824-412c-aef0-8befad6975bb-online-tls" (OuterVolumeSpecName: "online-tls") pod "4b0421c4-6824-412c-aef0-8befad6975bb" (UID: "4b0421c4-6824-412c-aef0-8befad6975bb"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 06:54:42.443474 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.443450 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:54:42.493048 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.492970 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a0305aa7-15c7-4cb8-8d49-96e4437444a6-registry-tls\") pod \"a0305aa7-15c7-4cb8-8d49-96e4437444a6\" (UID: \"a0305aa7-15c7-4cb8-8d49-96e4437444a6\") " Sep 09 06:54:42.493048 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.493010 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a0305aa7-15c7-4cb8-8d49-96e4437444a6-offline-tls\") pod \"a0305aa7-15c7-4cb8-8d49-96e4437444a6\" (UID: \"a0305aa7-15c7-4cb8-8d49-96e4437444a6\") " Sep 09 06:54:42.493264 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.493081 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a0305aa7-15c7-4cb8-8d49-96e4437444a6-feast-data\") pod \"a0305aa7-15c7-4cb8-8d49-96e4437444a6\" (UID: \"a0305aa7-15c7-4cb8-8d49-96e4437444a6\") " Sep 09 06:54:42.493264 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.493119 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a0305aa7-15c7-4cb8-8d49-96e4437444a6-ui-tls\") pod \"a0305aa7-15c7-4cb8-8d49-96e4437444a6\" (UID: \"a0305aa7-15c7-4cb8-8d49-96e4437444a6\") " Sep 09 06:54:42.493264 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.493149 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k56s4\" (UniqueName: \"kubernetes.io/projected/a0305aa7-15c7-4cb8-8d49-96e4437444a6-kube-api-access-k56s4\") pod \"a0305aa7-15c7-4cb8-8d49-96e4437444a6\" (UID: \"a0305aa7-15c7-4cb8-8d49-96e4437444a6\") " Sep 09 06:54:42.493264 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.493196 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a0305aa7-15c7-4cb8-8d49-96e4437444a6-online-tls\") pod \"a0305aa7-15c7-4cb8-8d49-96e4437444a6\" (UID: \"a0305aa7-15c7-4cb8-8d49-96e4437444a6\") " Sep 09 06:54:42.493478 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.493451 2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/4b0421c4-6824-412c-aef0-8befad6975bb-registry-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:54:42.493478 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.493475 2559 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4b0421c4-6824-412c-aef0-8befad6975bb-feast-data\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:54:42.493585 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.493488 2559 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4b0421c4-6824-412c-aef0-8befad6975bb-online-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:54:42.493585 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.493501 2559 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4b0421c4-6824-412c-aef0-8befad6975bb-offline-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:54:42.493585 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.493514 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87bdr\" (UniqueName: \"kubernetes.io/projected/4b0421c4-6824-412c-aef0-8befad6975bb-kube-api-access-87bdr\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:54:42.493585 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.493527 2559 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4b0421c4-6824-412c-aef0-8befad6975bb-ui-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:54:42.493798 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.493607 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0305aa7-15c7-4cb8-8d49-96e4437444a6-feast-data" (OuterVolumeSpecName: "feast-data") pod "a0305aa7-15c7-4cb8-8d49-96e4437444a6" (UID: "a0305aa7-15c7-4cb8-8d49-96e4437444a6"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 06:54:42.495403 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.495364 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0305aa7-15c7-4cb8-8d49-96e4437444a6-online-tls" (OuterVolumeSpecName: "online-tls") pod "a0305aa7-15c7-4cb8-8d49-96e4437444a6" (UID: "a0305aa7-15c7-4cb8-8d49-96e4437444a6"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 06:54:42.495403 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.495381 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0305aa7-15c7-4cb8-8d49-96e4437444a6-kube-api-access-k56s4" (OuterVolumeSpecName: "kube-api-access-k56s4") pod "a0305aa7-15c7-4cb8-8d49-96e4437444a6" (UID: "a0305aa7-15c7-4cb8-8d49-96e4437444a6"). InnerVolumeSpecName "kube-api-access-k56s4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 06:54:42.495541 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.495391 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0305aa7-15c7-4cb8-8d49-96e4437444a6-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "a0305aa7-15c7-4cb8-8d49-96e4437444a6" (UID: "a0305aa7-15c7-4cb8-8d49-96e4437444a6"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 06:54:42.495612 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.495594 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0305aa7-15c7-4cb8-8d49-96e4437444a6-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a0305aa7-15c7-4cb8-8d49-96e4437444a6" (UID: "a0305aa7-15c7-4cb8-8d49-96e4437444a6"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 06:54:42.495661 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.495646 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0305aa7-15c7-4cb8-8d49-96e4437444a6-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "a0305aa7-15c7-4cb8-8d49-96e4437444a6" (UID: "a0305aa7-15c7-4cb8-8d49-96e4437444a6"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 06:54:42.594132 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.594095 2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a0305aa7-15c7-4cb8-8d49-96e4437444a6-registry-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:54:42.594132 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.594123 2559 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a0305aa7-15c7-4cb8-8d49-96e4437444a6-offline-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:54:42.594132 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.594135 2559 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a0305aa7-15c7-4cb8-8d49-96e4437444a6-feast-data\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:54:42.594371 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.594146 2559 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a0305aa7-15c7-4cb8-8d49-96e4437444a6-ui-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:54:42.594371 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.594159 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k56s4\" (UniqueName: \"kubernetes.io/projected/a0305aa7-15c7-4cb8-8d49-96e4437444a6-kube-api-access-k56s4\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:54:42.594371 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.594171 2559 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a0305aa7-15c7-4cb8-8d49-96e4437444a6-online-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:54:42.800372 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.800291 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" event={"ID":"4b0421c4-6824-412c-aef0-8befad6975bb","Type":"ContainerDied","Data":"812654ea6ce257b189ceed08f23d6bfb70a6311135fd97c6fb81db960253c9ab"} Sep 09 06:54:42.800372 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.800306 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b" Sep 09 06:54:42.800372 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.800350 2559 scope.go:117] "RemoveContainer" containerID="749b5a0081d4cd295292a2bbe3d8eeb6cdf2c4bcff4ec98ca96586602e4d74f2" Sep 09 06:54:42.806035 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.805991 2559 generic.go:356] "Generic (PLEG): container finished" podID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerID="7b0537509c8551e842855534d3ff89cc22bd50a74ed0a6720c1e82767460090c" exitCode=137 Sep 09 06:54:42.806035 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.806015 2559 generic.go:356] "Generic (PLEG): container finished" podID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerID="d9f5ff234d530d3f5c240b5436b02a5f53968d3022b142922a98f6c087fd2de4" exitCode=137 Sep 09 06:54:42.806222 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.806106 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" event={"ID":"a0305aa7-15c7-4cb8-8d49-96e4437444a6","Type":"ContainerDied","Data":"7b0537509c8551e842855534d3ff89cc22bd50a74ed0a6720c1e82767460090c"} Sep 09 06:54:42.806222 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.806141 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" Sep 09 06:54:42.806222 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.806151 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" event={"ID":"a0305aa7-15c7-4cb8-8d49-96e4437444a6","Type":"ContainerDied","Data":"d9f5ff234d530d3f5c240b5436b02a5f53968d3022b142922a98f6c087fd2de4"} Sep 09 06:54:42.806222 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.806179 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp" event={"ID":"a0305aa7-15c7-4cb8-8d49-96e4437444a6","Type":"ContainerDied","Data":"025a9ffe435d8bbdfd6f205e9e67d0bc61bfa9e479ae8e8224782e82bea36ec2"} Sep 09 06:54:42.815148 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.815116 2559 scope.go:117] "RemoveContainer" containerID="ab5d1ed754e47bd4a353c91a2076e8aa09cf4114d1a68acb630dd24b27961f2b" Sep 09 06:54:42.821064 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.821029 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b"] Sep 09 06:54:42.823763 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.823741 2559 scope.go:117] "RemoveContainer" containerID="f13c9bba2972d6fe62603fdda086227f35da31041f3a3d7fbd6ab2e0320d7410" Sep 09 06:54:42.824493 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.824472 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-59f4c548f5-8tm6b"] Sep 09 06:54:42.831801 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.831777 2559 scope.go:117] "RemoveContainer" containerID="8ee3cc1da241178c6b3ea8ca2852c8b2f65f352022c1776bd2f27a7f753d13ba" Sep 09 06:54:42.837203 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.837182 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp"] Sep 09 06:54:42.839874 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.839854 2559 scope.go:117] "RemoveContainer" containerID="9ecd681b054116ac1726014b9212d2da21d2cea86e77b310b12e2883b8564aaf" Sep 09 06:54:42.842951 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.842931 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-84dccd7859-r7kgp"] Sep 09 06:54:42.847476 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.847461 2559 scope.go:117] "RemoveContainer" containerID="a26f6004d2eedabdde2d58edd6fc5b7d6bfc0d56da5f8cd146d1ae9a31143e2f" Sep 09 06:54:42.854285 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.854269 2559 scope.go:117] "RemoveContainer" containerID="7b0537509c8551e842855534d3ff89cc22bd50a74ed0a6720c1e82767460090c" Sep 09 06:54:42.861757 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.861738 2559 scope.go:117] "RemoveContainer" containerID="13c8e1848c6346ec154b75ad9c17423c99bb40fa11c97bdfbd4f9aa269242b2f" Sep 09 06:54:42.868757 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.868741 2559 scope.go:117] "RemoveContainer" containerID="d9f5ff234d530d3f5c240b5436b02a5f53968d3022b142922a98f6c087fd2de4" Sep 09 06:54:42.875672 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.875656 2559 scope.go:117] "RemoveContainer" containerID="13affb8e13c20d2eba15d18c46c188f439652fb225afe821f90ea7d4c0465389" Sep 09 06:54:42.882526 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.882510 2559 scope.go:117] "RemoveContainer" containerID="c966d7dc4798f2105ef58e5c73c69201475903283fe96b233f248f0b6e9e69a3" Sep 09 06:54:42.889511 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.889496 2559 scope.go:117] "RemoveContainer" containerID="a26f6004d2eedabdde2d58edd6fc5b7d6bfc0d56da5f8cd146d1ae9a31143e2f" Sep 09 06:54:42.889781 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:54:42.889760 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a26f6004d2eedabdde2d58edd6fc5b7d6bfc0d56da5f8cd146d1ae9a31143e2f\": container with ID starting with a26f6004d2eedabdde2d58edd6fc5b7d6bfc0d56da5f8cd146d1ae9a31143e2f not found: ID does not exist" containerID="a26f6004d2eedabdde2d58edd6fc5b7d6bfc0d56da5f8cd146d1ae9a31143e2f" Sep 09 06:54:42.889880 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.889790 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a26f6004d2eedabdde2d58edd6fc5b7d6bfc0d56da5f8cd146d1ae9a31143e2f"} err="failed to get container status \"a26f6004d2eedabdde2d58edd6fc5b7d6bfc0d56da5f8cd146d1ae9a31143e2f\": rpc error: code = NotFound desc = could not find container \"a26f6004d2eedabdde2d58edd6fc5b7d6bfc0d56da5f8cd146d1ae9a31143e2f\": container with ID starting with a26f6004d2eedabdde2d58edd6fc5b7d6bfc0d56da5f8cd146d1ae9a31143e2f not found: ID does not exist" Sep 09 06:54:42.889880 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.889808 2559 scope.go:117] "RemoveContainer" containerID="7b0537509c8551e842855534d3ff89cc22bd50a74ed0a6720c1e82767460090c" Sep 09 06:54:42.890086 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:54:42.890068 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b0537509c8551e842855534d3ff89cc22bd50a74ed0a6720c1e82767460090c\": container with ID starting with 7b0537509c8551e842855534d3ff89cc22bd50a74ed0a6720c1e82767460090c not found: ID does not exist" containerID="7b0537509c8551e842855534d3ff89cc22bd50a74ed0a6720c1e82767460090c" Sep 09 06:54:42.890132 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.890091 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b0537509c8551e842855534d3ff89cc22bd50a74ed0a6720c1e82767460090c"} err="failed to get container status \"7b0537509c8551e842855534d3ff89cc22bd50a74ed0a6720c1e82767460090c\": rpc error: code = NotFound desc = could not find container \"7b0537509c8551e842855534d3ff89cc22bd50a74ed0a6720c1e82767460090c\": container with ID starting with 7b0537509c8551e842855534d3ff89cc22bd50a74ed0a6720c1e82767460090c not found: ID does not exist" Sep 09 06:54:42.890132 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.890105 2559 scope.go:117] "RemoveContainer" containerID="13c8e1848c6346ec154b75ad9c17423c99bb40fa11c97bdfbd4f9aa269242b2f" Sep 09 06:54:42.890355 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:54:42.890338 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13c8e1848c6346ec154b75ad9c17423c99bb40fa11c97bdfbd4f9aa269242b2f\": container with ID starting with 13c8e1848c6346ec154b75ad9c17423c99bb40fa11c97bdfbd4f9aa269242b2f not found: ID does not exist" containerID="13c8e1848c6346ec154b75ad9c17423c99bb40fa11c97bdfbd4f9aa269242b2f" Sep 09 06:54:42.890416 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.890358 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13c8e1848c6346ec154b75ad9c17423c99bb40fa11c97bdfbd4f9aa269242b2f"} err="failed to get container status \"13c8e1848c6346ec154b75ad9c17423c99bb40fa11c97bdfbd4f9aa269242b2f\": rpc error: code = NotFound desc = could not find container \"13c8e1848c6346ec154b75ad9c17423c99bb40fa11c97bdfbd4f9aa269242b2f\": container with ID starting with 13c8e1848c6346ec154b75ad9c17423c99bb40fa11c97bdfbd4f9aa269242b2f not found: ID does not exist" Sep 09 06:54:42.890416 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.890370 2559 scope.go:117] "RemoveContainer" containerID="d9f5ff234d530d3f5c240b5436b02a5f53968d3022b142922a98f6c087fd2de4" Sep 09 06:54:42.890552 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:54:42.890536 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9f5ff234d530d3f5c240b5436b02a5f53968d3022b142922a98f6c087fd2de4\": container with ID starting with d9f5ff234d530d3f5c240b5436b02a5f53968d3022b142922a98f6c087fd2de4 not found: ID does not exist" containerID="d9f5ff234d530d3f5c240b5436b02a5f53968d3022b142922a98f6c087fd2de4" Sep 09 06:54:42.890603 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.890556 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9f5ff234d530d3f5c240b5436b02a5f53968d3022b142922a98f6c087fd2de4"} err="failed to get container status \"d9f5ff234d530d3f5c240b5436b02a5f53968d3022b142922a98f6c087fd2de4\": rpc error: code = NotFound desc = could not find container \"d9f5ff234d530d3f5c240b5436b02a5f53968d3022b142922a98f6c087fd2de4\": container with ID starting with d9f5ff234d530d3f5c240b5436b02a5f53968d3022b142922a98f6c087fd2de4 not found: ID does not exist" Sep 09 06:54:42.890603 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.890568 2559 scope.go:117] "RemoveContainer" containerID="13affb8e13c20d2eba15d18c46c188f439652fb225afe821f90ea7d4c0465389" Sep 09 06:54:42.890778 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:54:42.890762 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13affb8e13c20d2eba15d18c46c188f439652fb225afe821f90ea7d4c0465389\": container with ID starting with 13affb8e13c20d2eba15d18c46c188f439652fb225afe821f90ea7d4c0465389 not found: ID does not exist" containerID="13affb8e13c20d2eba15d18c46c188f439652fb225afe821f90ea7d4c0465389" Sep 09 06:54:42.890836 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.890780 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13affb8e13c20d2eba15d18c46c188f439652fb225afe821f90ea7d4c0465389"} err="failed to get container status \"13affb8e13c20d2eba15d18c46c188f439652fb225afe821f90ea7d4c0465389\": rpc error: code = NotFound desc = could not find container \"13affb8e13c20d2eba15d18c46c188f439652fb225afe821f90ea7d4c0465389\": container with ID starting with 13affb8e13c20d2eba15d18c46c188f439652fb225afe821f90ea7d4c0465389 not found: ID does not exist" Sep 09 06:54:42.890836 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.890793 2559 scope.go:117] "RemoveContainer" containerID="c966d7dc4798f2105ef58e5c73c69201475903283fe96b233f248f0b6e9e69a3" Sep 09 06:54:42.891020 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:54:42.891001 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c966d7dc4798f2105ef58e5c73c69201475903283fe96b233f248f0b6e9e69a3\": container with ID starting with c966d7dc4798f2105ef58e5c73c69201475903283fe96b233f248f0b6e9e69a3 not found: ID does not exist" containerID="c966d7dc4798f2105ef58e5c73c69201475903283fe96b233f248f0b6e9e69a3" Sep 09 06:54:42.891105 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.891029 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c966d7dc4798f2105ef58e5c73c69201475903283fe96b233f248f0b6e9e69a3"} err="failed to get container status \"c966d7dc4798f2105ef58e5c73c69201475903283fe96b233f248f0b6e9e69a3\": rpc error: code = NotFound desc = could not find container \"c966d7dc4798f2105ef58e5c73c69201475903283fe96b233f248f0b6e9e69a3\": container with ID starting with c966d7dc4798f2105ef58e5c73c69201475903283fe96b233f248f0b6e9e69a3 not found: ID does not exist" Sep 09 06:54:42.891178 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.891165 2559 scope.go:117] "RemoveContainer" containerID="a26f6004d2eedabdde2d58edd6fc5b7d6bfc0d56da5f8cd146d1ae9a31143e2f" Sep 09 06:54:42.891400 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.891382 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a26f6004d2eedabdde2d58edd6fc5b7d6bfc0d56da5f8cd146d1ae9a31143e2f"} err="failed to get container status \"a26f6004d2eedabdde2d58edd6fc5b7d6bfc0d56da5f8cd146d1ae9a31143e2f\": rpc error: code = NotFound desc = could not find container \"a26f6004d2eedabdde2d58edd6fc5b7d6bfc0d56da5f8cd146d1ae9a31143e2f\": container with ID starting with a26f6004d2eedabdde2d58edd6fc5b7d6bfc0d56da5f8cd146d1ae9a31143e2f not found: ID does not exist" Sep 09 06:54:42.891474 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.891401 2559 scope.go:117] "RemoveContainer" containerID="7b0537509c8551e842855534d3ff89cc22bd50a74ed0a6720c1e82767460090c" Sep 09 06:54:42.891623 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.891605 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b0537509c8551e842855534d3ff89cc22bd50a74ed0a6720c1e82767460090c"} err="failed to get container status \"7b0537509c8551e842855534d3ff89cc22bd50a74ed0a6720c1e82767460090c\": rpc error: code = NotFound desc = could not find container \"7b0537509c8551e842855534d3ff89cc22bd50a74ed0a6720c1e82767460090c\": container with ID starting with 7b0537509c8551e842855534d3ff89cc22bd50a74ed0a6720c1e82767460090c not found: ID does not exist" Sep 09 06:54:42.891623 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.891623 2559 scope.go:117] "RemoveContainer" containerID="13c8e1848c6346ec154b75ad9c17423c99bb40fa11c97bdfbd4f9aa269242b2f" Sep 09 06:54:42.891828 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.891812 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13c8e1848c6346ec154b75ad9c17423c99bb40fa11c97bdfbd4f9aa269242b2f"} err="failed to get container status \"13c8e1848c6346ec154b75ad9c17423c99bb40fa11c97bdfbd4f9aa269242b2f\": rpc error: code = NotFound desc = could not find container \"13c8e1848c6346ec154b75ad9c17423c99bb40fa11c97bdfbd4f9aa269242b2f\": container with ID starting with 13c8e1848c6346ec154b75ad9c17423c99bb40fa11c97bdfbd4f9aa269242b2f not found: ID does not exist" Sep 09 06:54:42.891872 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.891829 2559 scope.go:117] "RemoveContainer" containerID="d9f5ff234d530d3f5c240b5436b02a5f53968d3022b142922a98f6c087fd2de4" Sep 09 06:54:42.892023 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.892006 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9f5ff234d530d3f5c240b5436b02a5f53968d3022b142922a98f6c087fd2de4"} err="failed to get container status \"d9f5ff234d530d3f5c240b5436b02a5f53968d3022b142922a98f6c087fd2de4\": rpc error: code = NotFound desc = could not find container \"d9f5ff234d530d3f5c240b5436b02a5f53968d3022b142922a98f6c087fd2de4\": container with ID starting with d9f5ff234d530d3f5c240b5436b02a5f53968d3022b142922a98f6c087fd2de4 not found: ID does not exist" Sep 09 06:54:42.892088 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.892023 2559 scope.go:117] "RemoveContainer" containerID="13affb8e13c20d2eba15d18c46c188f439652fb225afe821f90ea7d4c0465389" Sep 09 06:54:42.892266 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.892248 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13affb8e13c20d2eba15d18c46c188f439652fb225afe821f90ea7d4c0465389"} err="failed to get container status \"13affb8e13c20d2eba15d18c46c188f439652fb225afe821f90ea7d4c0465389\": rpc error: code = NotFound desc = could not find container \"13affb8e13c20d2eba15d18c46c188f439652fb225afe821f90ea7d4c0465389\": container with ID starting with 13affb8e13c20d2eba15d18c46c188f439652fb225afe821f90ea7d4c0465389 not found: ID does not exist" Sep 09 06:54:42.892316 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.892267 2559 scope.go:117] "RemoveContainer" containerID="c966d7dc4798f2105ef58e5c73c69201475903283fe96b233f248f0b6e9e69a3" Sep 09 06:54:42.892549 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:42.892530 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c966d7dc4798f2105ef58e5c73c69201475903283fe96b233f248f0b6e9e69a3"} err="failed to get container status \"c966d7dc4798f2105ef58e5c73c69201475903283fe96b233f248f0b6e9e69a3\": rpc error: code = NotFound desc = could not find container \"c966d7dc4798f2105ef58e5c73c69201475903283fe96b233f248f0b6e9e69a3\": container with ID starting with c966d7dc4798f2105ef58e5c73c69201475903283fe96b233f248f0b6e9e69a3 not found: ID does not exist" Sep 09 06:54:44.701682 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:44.701650 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b0421c4-6824-412c-aef0-8befad6975bb" path="/var/lib/kubelet/pods/4b0421c4-6824-412c-aef0-8befad6975bb/volumes" Sep 09 06:54:44.702210 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:44.702196 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" path="/var/lib/kubelet/pods/a0305aa7-15c7-4cb8-8d49-96e4437444a6/volumes" Sep 09 06:54:54.617937 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.617903 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/postgres-864846d4f5-xxqvd"] Sep 09 06:54:54.618451 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.618432 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerName="ui" Sep 09 06:54:54.618496 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.618455 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerName="ui" Sep 09 06:54:54.618496 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.618476 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b0421c4-6824-412c-aef0-8befad6975bb" containerName="online" Sep 09 06:54:54.618496 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.618487 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b0421c4-6824-412c-aef0-8befad6975bb" containerName="online" Sep 09 06:54:54.618596 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.618501 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerName="offline" Sep 09 06:54:54.618596 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.618511 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerName="offline" Sep 09 06:54:54.618596 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.618525 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerName="registry" Sep 09 06:54:54.618596 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.618534 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerName="registry" Sep 09 06:54:54.618596 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.618544 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b0421c4-6824-412c-aef0-8befad6975bb" containerName="offline" Sep 09 06:54:54.618596 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.618553 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b0421c4-6824-412c-aef0-8befad6975bb" containerName="offline" Sep 09 06:54:54.618596 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.618570 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b0421c4-6824-412c-aef0-8befad6975bb" containerName="feast-apply" Sep 09 06:54:54.618596 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.618578 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b0421c4-6824-412c-aef0-8befad6975bb" containerName="feast-apply" Sep 09 06:54:54.618596 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.618587 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerName="feast-init" Sep 09 06:54:54.618596 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.618594 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerName="feast-init" Sep 09 06:54:54.618925 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.618609 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerName="feast-apply" Sep 09 06:54:54.618925 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.618618 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerName="feast-apply" Sep 09 06:54:54.618925 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.618631 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerName="online" Sep 09 06:54:54.618925 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.618640 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerName="online" Sep 09 06:54:54.618925 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.618650 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b0421c4-6824-412c-aef0-8befad6975bb" containerName="ui" Sep 09 06:54:54.618925 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.618658 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b0421c4-6824-412c-aef0-8befad6975bb" containerName="ui" Sep 09 06:54:54.618925 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.618670 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b0421c4-6824-412c-aef0-8befad6975bb" containerName="feast-init" Sep 09 06:54:54.618925 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.618678 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b0421c4-6824-412c-aef0-8befad6975bb" containerName="feast-init" Sep 09 06:54:54.618925 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.618750 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerName="online" Sep 09 06:54:54.618925 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.618762 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b0421c4-6824-412c-aef0-8befad6975bb" containerName="offline" Sep 09 06:54:54.618925 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.618772 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerName="offline" Sep 09 06:54:54.618925 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.618781 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerName="ui" Sep 09 06:54:54.618925 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.618790 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b0421c4-6824-412c-aef0-8befad6975bb" containerName="ui" Sep 09 06:54:54.618925 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.618800 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0305aa7-15c7-4cb8-8d49-96e4437444a6" containerName="registry" Sep 09 06:54:54.618925 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.618808 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b0421c4-6824-412c-aef0-8befad6975bb" containerName="online" Sep 09 06:54:54.621763 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.621747 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-xxqvd" Sep 09 06:54:54.624008 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.623957 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\"" Sep 09 06:54:54.624178 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.624097 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-r54j4\"" Sep 09 06:54:54.624260 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.624192 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 09 06:54:54.624839 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.624815 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 09 06:54:54.628346 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.628325 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-xxqvd"] Sep 09 06:54:54.658184 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.658152 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/redis-69bbd798b6-frltq"] Sep 09 06:54:54.661229 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.661208 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-frltq" Sep 09 06:54:54.668919 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.668799 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-frltq"] Sep 09 06:54:54.693191 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.693168 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/24fc2455-9c8e-42eb-a853-125632a7b0bc-postgresdata\") pod \"postgres-864846d4f5-xxqvd\" (UID: \"24fc2455-9c8e-42eb-a853-125632a7b0bc\") " pod="test-ns-feast/postgres-864846d4f5-xxqvd" Sep 09 06:54:54.693290 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.693230 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82tbz\" (UniqueName: \"kubernetes.io/projected/24fc2455-9c8e-42eb-a853-125632a7b0bc-kube-api-access-82tbz\") pod \"postgres-864846d4f5-xxqvd\" (UID: \"24fc2455-9c8e-42eb-a853-125632a7b0bc\") " pod="test-ns-feast/postgres-864846d4f5-xxqvd" Sep 09 06:54:54.793820 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.793783 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwqvf\" (UniqueName: \"kubernetes.io/projected/855f0fe2-b3f4-4541-b4ce-55c507a3356e-kube-api-access-mwqvf\") pod \"redis-69bbd798b6-frltq\" (UID: \"855f0fe2-b3f4-4541-b4ce-55c507a3356e\") " pod="test-ns-feast/redis-69bbd798b6-frltq" Sep 09 06:54:54.793820 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.793825 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/24fc2455-9c8e-42eb-a853-125632a7b0bc-postgresdata\") pod \"postgres-864846d4f5-xxqvd\" (UID: \"24fc2455-9c8e-42eb-a853-125632a7b0bc\") " pod="test-ns-feast/postgres-864846d4f5-xxqvd" Sep 09 06:54:54.794022 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.793861 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82tbz\" (UniqueName: \"kubernetes.io/projected/24fc2455-9c8e-42eb-a853-125632a7b0bc-kube-api-access-82tbz\") pod \"postgres-864846d4f5-xxqvd\" (UID: \"24fc2455-9c8e-42eb-a853-125632a7b0bc\") " pod="test-ns-feast/postgres-864846d4f5-xxqvd" Sep 09 06:54:54.794216 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.794195 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/24fc2455-9c8e-42eb-a853-125632a7b0bc-postgresdata\") pod \"postgres-864846d4f5-xxqvd\" (UID: \"24fc2455-9c8e-42eb-a853-125632a7b0bc\") " pod="test-ns-feast/postgres-864846d4f5-xxqvd" Sep 09 06:54:54.801910 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.801882 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82tbz\" (UniqueName: \"kubernetes.io/projected/24fc2455-9c8e-42eb-a853-125632a7b0bc-kube-api-access-82tbz\") pod \"postgres-864846d4f5-xxqvd\" (UID: \"24fc2455-9c8e-42eb-a853-125632a7b0bc\") " pod="test-ns-feast/postgres-864846d4f5-xxqvd" Sep 09 06:54:54.894888 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.894815 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwqvf\" (UniqueName: \"kubernetes.io/projected/855f0fe2-b3f4-4541-b4ce-55c507a3356e-kube-api-access-mwqvf\") pod \"redis-69bbd798b6-frltq\" (UID: \"855f0fe2-b3f4-4541-b4ce-55c507a3356e\") " pod="test-ns-feast/redis-69bbd798b6-frltq" Sep 09 06:54:54.902446 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.902421 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwqvf\" (UniqueName: \"kubernetes.io/projected/855f0fe2-b3f4-4541-b4ce-55c507a3356e-kube-api-access-mwqvf\") pod \"redis-69bbd798b6-frltq\" (UID: \"855f0fe2-b3f4-4541-b4ce-55c507a3356e\") " pod="test-ns-feast/redis-69bbd798b6-frltq" Sep 09 06:54:54.943504 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.943476 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-xxqvd" Sep 09 06:54:54.970590 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:54.970560 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-frltq" Sep 09 06:54:55.217905 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:55.217763 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-xxqvd"] Sep 09 06:54:55.220571 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:54:55.220543 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod24fc2455_9c8e_42eb_a853_125632a7b0bc.slice/crio-dd763cb2a77415b2dfd7358e7ac789a04b0cc76a0e1c95acd04e47e5e5db3849 WatchSource:0}: Error finding container dd763cb2a77415b2dfd7358e7ac789a04b0cc76a0e1c95acd04e47e5e5db3849: Status 404 returned error can't find the container with id dd763cb2a77415b2dfd7358e7ac789a04b0cc76a0e1c95acd04e47e5e5db3849 Sep 09 06:54:55.222324 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:55.222306 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 06:54:55.247519 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:55.247497 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-frltq"] Sep 09 06:54:55.249848 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:54:55.249823 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod855f0fe2_b3f4_4541_b4ce_55c507a3356e.slice/crio-7eb2b726f9660066a3f4aba5d70d1df3b744466dec90bbfb4b4a806c91d9721b WatchSource:0}: Error finding container 7eb2b726f9660066a3f4aba5d70d1df3b744466dec90bbfb4b4a806c91d9721b: Status 404 returned error can't find the container with id 7eb2b726f9660066a3f4aba5d70d1df3b744466dec90bbfb4b4a806c91d9721b Sep 09 06:54:55.870846 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:55.870807 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-frltq" event={"ID":"855f0fe2-b3f4-4541-b4ce-55c507a3356e","Type":"ContainerStarted","Data":"7eb2b726f9660066a3f4aba5d70d1df3b744466dec90bbfb4b4a806c91d9721b"} Sep 09 06:54:55.872000 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:55.871973 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-xxqvd" event={"ID":"24fc2455-9c8e-42eb-a853-125632a7b0bc","Type":"ContainerStarted","Data":"dd763cb2a77415b2dfd7358e7ac789a04b0cc76a0e1c95acd04e47e5e5db3849"} Sep 09 06:54:57.885201 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:57.885158 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-frltq" event={"ID":"855f0fe2-b3f4-4541-b4ce-55c507a3356e","Type":"ContainerStarted","Data":"3f2359588988b12e35a004dcd8bd5ba4fa2d70e9df160570e4dd263154b12f3e"} Sep 09 06:54:57.898977 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:57.898922 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/redis-69bbd798b6-frltq" podStartSLOduration=2.223006588 podStartE2EDuration="3.898902852s" totalImagesPullingTime="1.675896264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:54:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:54:57.898356353 +0000 UTC m=+995.794773777" watchObservedRunningTime="2026-09-09 06:54:57.898902852 +0000 UTC m=+995.795320277" Sep 09 06:54:59.897365 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:59.897330 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-xxqvd" event={"ID":"24fc2455-9c8e-42eb-a853-125632a7b0bc","Type":"ContainerStarted","Data":"f290dc2b6bd40561f636dc1753d789221afb68e6c726b2e6f66c849a9a241cb2"} Sep 09 06:54:59.912704 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:54:59.912657 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/postgres-864846d4f5-xxqvd" podStartSLOduration=1.395750134 podStartE2EDuration="5.912642638s" totalImagesPullingTime="4.516892504s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:54:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:54:59.912079566 +0000 UTC m=+997.808496988" watchObservedRunningTime="2026-09-09 06:54:59.912642638 +0000 UTC m=+997.809060059" Sep 09 06:55:03.431006 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:03.430968 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw"] Sep 09 06:55:03.434810 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:03.434790 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" Sep 09 06:55:03.437101 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:03.437079 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-registry-tls\"" Sep 09 06:55:03.437483 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:03.437467 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-offline-tls\"" Sep 09 06:55:03.438196 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:03.438176 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-online-tls\"" Sep 09 06:55:03.438410 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:03.438395 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-52522\"" Sep 09 06:55:03.438463 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:03.438416 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Sep 09 06:55:03.451912 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:03.451891 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw"] Sep 09 06:55:03.573452 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:03.573421 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e36dcba8-15e0-4688-8eb1-f465a54feb3d-offline-tls\") pod \"feast-credit-scoring-7bf7ccdbb-6h9jw\" (UID: \"e36dcba8-15e0-4688-8eb1-f465a54feb3d\") " pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" Sep 09 06:55:03.573606 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:03.573474 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kh82m\" (UniqueName: \"kubernetes.io/projected/e36dcba8-15e0-4688-8eb1-f465a54feb3d-kube-api-access-kh82m\") pod \"feast-credit-scoring-7bf7ccdbb-6h9jw\" (UID: \"e36dcba8-15e0-4688-8eb1-f465a54feb3d\") " pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" Sep 09 06:55:03.573606 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:03.573523 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e36dcba8-15e0-4688-8eb1-f465a54feb3d-online-tls\") pod \"feast-credit-scoring-7bf7ccdbb-6h9jw\" (UID: \"e36dcba8-15e0-4688-8eb1-f465a54feb3d\") " pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" Sep 09 06:55:03.573606 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:03.573568 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e36dcba8-15e0-4688-8eb1-f465a54feb3d-registry-tls\") pod \"feast-credit-scoring-7bf7ccdbb-6h9jw\" (UID: \"e36dcba8-15e0-4688-8eb1-f465a54feb3d\") " pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" Sep 09 06:55:03.573606 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:03.573600 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e36dcba8-15e0-4688-8eb1-f465a54feb3d-feast-data\") pod \"feast-credit-scoring-7bf7ccdbb-6h9jw\" (UID: \"e36dcba8-15e0-4688-8eb1-f465a54feb3d\") " pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" Sep 09 06:55:03.675048 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:03.675018 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e36dcba8-15e0-4688-8eb1-f465a54feb3d-feast-data\") pod \"feast-credit-scoring-7bf7ccdbb-6h9jw\" (UID: \"e36dcba8-15e0-4688-8eb1-f465a54feb3d\") " pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" Sep 09 06:55:03.675230 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:03.675088 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e36dcba8-15e0-4688-8eb1-f465a54feb3d-offline-tls\") pod \"feast-credit-scoring-7bf7ccdbb-6h9jw\" (UID: \"e36dcba8-15e0-4688-8eb1-f465a54feb3d\") " pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" Sep 09 06:55:03.675230 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:03.675134 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kh82m\" (UniqueName: \"kubernetes.io/projected/e36dcba8-15e0-4688-8eb1-f465a54feb3d-kube-api-access-kh82m\") pod \"feast-credit-scoring-7bf7ccdbb-6h9jw\" (UID: \"e36dcba8-15e0-4688-8eb1-f465a54feb3d\") " pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" Sep 09 06:55:03.675230 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:03.675154 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e36dcba8-15e0-4688-8eb1-f465a54feb3d-online-tls\") pod \"feast-credit-scoring-7bf7ccdbb-6h9jw\" (UID: \"e36dcba8-15e0-4688-8eb1-f465a54feb3d\") " pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" Sep 09 06:55:03.675230 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:03.675193 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e36dcba8-15e0-4688-8eb1-f465a54feb3d-registry-tls\") pod \"feast-credit-scoring-7bf7ccdbb-6h9jw\" (UID: \"e36dcba8-15e0-4688-8eb1-f465a54feb3d\") " pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" Sep 09 06:55:03.675396 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:55:03.675253 2559 secret.go:189] Couldn't get secret test-ns-feast/feast-credit-scoring-offline-tls: secret "feast-credit-scoring-offline-tls" not found Sep 09 06:55:03.675396 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:55:03.675324 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e36dcba8-15e0-4688-8eb1-f465a54feb3d-offline-tls podName:e36dcba8-15e0-4688-8eb1-f465a54feb3d nodeName:}" failed. No retries permitted until 2026-09-09 06:55:04.175302836 +0000 UTC m=+1002.071720251 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/e36dcba8-15e0-4688-8eb1-f465a54feb3d-offline-tls") pod "feast-credit-scoring-7bf7ccdbb-6h9jw" (UID: "e36dcba8-15e0-4688-8eb1-f465a54feb3d") : secret "feast-credit-scoring-offline-tls" not found Sep 09 06:55:03.675476 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:03.675455 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e36dcba8-15e0-4688-8eb1-f465a54feb3d-feast-data\") pod \"feast-credit-scoring-7bf7ccdbb-6h9jw\" (UID: \"e36dcba8-15e0-4688-8eb1-f465a54feb3d\") " pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" Sep 09 06:55:03.677510 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:03.677484 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e36dcba8-15e0-4688-8eb1-f465a54feb3d-registry-tls\") pod \"feast-credit-scoring-7bf7ccdbb-6h9jw\" (UID: \"e36dcba8-15e0-4688-8eb1-f465a54feb3d\") " pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" Sep 09 06:55:03.677611 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:03.677520 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e36dcba8-15e0-4688-8eb1-f465a54feb3d-online-tls\") pod \"feast-credit-scoring-7bf7ccdbb-6h9jw\" (UID: \"e36dcba8-15e0-4688-8eb1-f465a54feb3d\") " pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" Sep 09 06:55:03.685587 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:03.685532 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kh82m\" (UniqueName: \"kubernetes.io/projected/e36dcba8-15e0-4688-8eb1-f465a54feb3d-kube-api-access-kh82m\") pod \"feast-credit-scoring-7bf7ccdbb-6h9jw\" (UID: \"e36dcba8-15e0-4688-8eb1-f465a54feb3d\") " pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" Sep 09 06:55:04.180768 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:04.180734 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e36dcba8-15e0-4688-8eb1-f465a54feb3d-offline-tls\") pod \"feast-credit-scoring-7bf7ccdbb-6h9jw\" (UID: \"e36dcba8-15e0-4688-8eb1-f465a54feb3d\") " pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" Sep 09 06:55:04.183255 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:04.183234 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e36dcba8-15e0-4688-8eb1-f465a54feb3d-offline-tls\") pod \"feast-credit-scoring-7bf7ccdbb-6h9jw\" (UID: \"e36dcba8-15e0-4688-8eb1-f465a54feb3d\") " pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" Sep 09 06:55:04.343984 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:04.343941 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" Sep 09 06:55:04.623826 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:04.623802 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw"] Sep 09 06:55:04.625204 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:55:04.625179 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode36dcba8_15e0_4688_8eb1_f465a54feb3d.slice/crio-547f9b9be1970b188b9aabb048294507aa3818c52483a77f55b15019c80d54ea WatchSource:0}: Error finding container 547f9b9be1970b188b9aabb048294507aa3818c52483a77f55b15019c80d54ea: Status 404 returned error can't find the container with id 547f9b9be1970b188b9aabb048294507aa3818c52483a77f55b15019c80d54ea Sep 09 06:55:04.920783 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:04.920703 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" event={"ID":"e36dcba8-15e0-4688-8eb1-f465a54feb3d","Type":"ContainerStarted","Data":"92e8ef2967a6f0918132fda6a5ebe32ca58fcbc158114a8ee0ac651ec5e2049d"} Sep 09 06:55:04.920783 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:04.920761 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" event={"ID":"e36dcba8-15e0-4688-8eb1-f465a54feb3d","Type":"ContainerStarted","Data":"547f9b9be1970b188b9aabb048294507aa3818c52483a77f55b15019c80d54ea"} Sep 09 06:55:05.927379 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:05.927344 2559 generic.go:356] "Generic (PLEG): container finished" podID="e36dcba8-15e0-4688-8eb1-f465a54feb3d" containerID="92e8ef2967a6f0918132fda6a5ebe32ca58fcbc158114a8ee0ac651ec5e2049d" exitCode=0 Sep 09 06:55:05.927783 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:05.927391 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" event={"ID":"e36dcba8-15e0-4688-8eb1-f465a54feb3d","Type":"ContainerDied","Data":"92e8ef2967a6f0918132fda6a5ebe32ca58fcbc158114a8ee0ac651ec5e2049d"} Sep 09 06:55:06.934251 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:06.934212 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" event={"ID":"e36dcba8-15e0-4688-8eb1-f465a54feb3d","Type":"ContainerStarted","Data":"6da4f5bf2aacb9b3cd028d27e7002943c974a4272ae1fe40d093ac623709f508"} Sep 09 06:55:12.965951 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:12.965920 2559 generic.go:356] "Generic (PLEG): container finished" podID="e36dcba8-15e0-4688-8eb1-f465a54feb3d" containerID="6da4f5bf2aacb9b3cd028d27e7002943c974a4272ae1fe40d093ac623709f508" exitCode=0 Sep 09 06:55:12.966377 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:12.965957 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" event={"ID":"e36dcba8-15e0-4688-8eb1-f465a54feb3d","Type":"ContainerDied","Data":"6da4f5bf2aacb9b3cd028d27e7002943c974a4272ae1fe40d093ac623709f508"} Sep 09 06:55:13.977754 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:13.977710 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" event={"ID":"e36dcba8-15e0-4688-8eb1-f465a54feb3d","Type":"ContainerStarted","Data":"fd22129d13ea7327bddd034364d4d4de20b439c7fff338cb1c3784d1176c06b2"} Sep 09 06:55:13.978259 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:13.977790 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" event={"ID":"e36dcba8-15e0-4688-8eb1-f465a54feb3d","Type":"ContainerStarted","Data":"2c84cade123a5c9f70ac306dd5d5e16469bdeac0b3468184856d668044d24af6"} Sep 09 06:55:13.978259 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:13.977826 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" event={"ID":"e36dcba8-15e0-4688-8eb1-f465a54feb3d","Type":"ContainerStarted","Data":"4caeb8088596cab38dca1ec988d29b81434e1aaabc8f8f038aafe20f43bc7c59"} Sep 09 06:55:14.004200 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:14.004136 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" podStartSLOduration=11.004112701 podStartE2EDuration="11.004112701s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:55:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:55:14.002481067 +0000 UTC m=+1011.898898488" watchObservedRunningTime="2026-09-09 06:55:14.004112701 +0000 UTC m=+1011.900530123" Sep 09 06:55:14.344407 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:14.344317 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" Sep 09 06:55:14.344407 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:14.344398 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" Sep 09 06:55:14.344594 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:14.344427 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" Sep 09 06:55:16.344866 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:16.344811 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" Sep 09 06:55:16.345266 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:16.344905 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" Sep 09 06:55:16.345266 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:16.344932 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" Sep 09 06:55:16.346691 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:16.346654 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" podUID="e36dcba8-15e0-4688-8eb1-f465a54feb3d" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.35:6571: connect: connection refused" Sep 09 06:55:16.346691 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:16.346671 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" podUID="e36dcba8-15e0-4688-8eb1-f465a54feb3d" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.35:8816: connect: connection refused" Sep 09 06:55:16.346867 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:16.346671 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" podUID="e36dcba8-15e0-4688-8eb1-f465a54feb3d" containerName="online" probeResult="failure" output="Get \"https://10.132.0.35:6567/health\": dial tcp 10.132.0.35:6567: connect: connection refused" Sep 09 06:55:19.344726 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:19.344674 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" podUID="e36dcba8-15e0-4688-8eb1-f465a54feb3d" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.35:6571: connect: connection refused" Sep 09 06:55:19.345260 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:19.344673 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" podUID="e36dcba8-15e0-4688-8eb1-f465a54feb3d" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.35:8816: connect: connection refused" Sep 09 06:55:19.345458 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:19.345428 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" podUID="e36dcba8-15e0-4688-8eb1-f465a54feb3d" containerName="online" probeResult="failure" output="Get \"https://10.132.0.35:6567/health\": dial tcp 10.132.0.35:6567: connect: connection refused" Sep 09 06:55:22.345023 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:22.344982 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" podUID="e36dcba8-15e0-4688-8eb1-f465a54feb3d" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.35:6571: connect: connection refused" Sep 09 06:55:22.345650 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:22.345621 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" Sep 09 06:55:22.346559 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:22.346535 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" Sep 09 06:55:22.351155 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:22.351134 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" Sep 09 06:55:23.034470 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:23.034437 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" Sep 09 06:55:25.344860 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:25.344820 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" podUID="e36dcba8-15e0-4688-8eb1-f465a54feb3d" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.35:6571: connect: connection refused" Sep 09 06:55:28.345695 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:28.345661 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" Sep 09 06:55:28.346483 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:55:28.346457 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" Sep 09 06:56:31.409995 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:31.409960 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw"] Sep 09 06:56:31.410582 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:31.410215 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" podUID="e36dcba8-15e0-4688-8eb1-f465a54feb3d" containerName="registry" containerID="cri-o://4caeb8088596cab38dca1ec988d29b81434e1aaabc8f8f038aafe20f43bc7c59" gracePeriod=30 Sep 09 06:56:31.410582 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:31.410277 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" podUID="e36dcba8-15e0-4688-8eb1-f465a54feb3d" containerName="offline" containerID="cri-o://fd22129d13ea7327bddd034364d4d4de20b439c7fff338cb1c3784d1176c06b2" gracePeriod=30 Sep 09 06:56:31.410582 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:31.410286 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" podUID="e36dcba8-15e0-4688-8eb1-f465a54feb3d" containerName="online" containerID="cri-o://2c84cade123a5c9f70ac306dd5d5e16469bdeac0b3468184856d668044d24af6" gracePeriod=30 Sep 09 06:56:31.426930 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:31.426902 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-xxqvd"] Sep 09 06:56:31.427060 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:31.427036 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/postgres-864846d4f5-xxqvd" podUID="24fc2455-9c8e-42eb-a853-125632a7b0bc" containerName="postgres" containerID="cri-o://f290dc2b6bd40561f636dc1753d789221afb68e6c726b2e6f66c849a9a241cb2" gracePeriod=30 Sep 09 06:56:31.438252 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:31.438228 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-frltq"] Sep 09 06:56:31.438403 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:31.438384 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/redis-69bbd798b6-frltq" podUID="855f0fe2-b3f4-4541-b4ce-55c507a3356e" containerName="redis" containerID="cri-o://3f2359588988b12e35a004dcd8bd5ba4fa2d70e9df160570e4dd263154b12f3e" gracePeriod=30 Sep 09 06:56:31.699032 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:31.699011 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-frltq" Sep 09 06:56:31.701879 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:31.701859 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-xxqvd" Sep 09 06:56:31.740287 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:31.740229 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/24fc2455-9c8e-42eb-a853-125632a7b0bc-postgresdata\") pod \"24fc2455-9c8e-42eb-a853-125632a7b0bc\" (UID: \"24fc2455-9c8e-42eb-a853-125632a7b0bc\") " Sep 09 06:56:31.741580 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:31.740496 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82tbz\" (UniqueName: \"kubernetes.io/projected/24fc2455-9c8e-42eb-a853-125632a7b0bc-kube-api-access-82tbz\") pod \"24fc2455-9c8e-42eb-a853-125632a7b0bc\" (UID: \"24fc2455-9c8e-42eb-a853-125632a7b0bc\") " Sep 09 06:56:31.741580 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:31.740580 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mwqvf\" (UniqueName: \"kubernetes.io/projected/855f0fe2-b3f4-4541-b4ce-55c507a3356e-kube-api-access-mwqvf\") pod \"855f0fe2-b3f4-4541-b4ce-55c507a3356e\" (UID: \"855f0fe2-b3f4-4541-b4ce-55c507a3356e\") " Sep 09 06:56:31.752838 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:31.752799 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/855f0fe2-b3f4-4541-b4ce-55c507a3356e-kube-api-access-mwqvf" (OuterVolumeSpecName: "kube-api-access-mwqvf") pod "855f0fe2-b3f4-4541-b4ce-55c507a3356e" (UID: "855f0fe2-b3f4-4541-b4ce-55c507a3356e"). InnerVolumeSpecName "kube-api-access-mwqvf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 06:56:31.753730 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:31.753705 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24fc2455-9c8e-42eb-a853-125632a7b0bc-kube-api-access-82tbz" (OuterVolumeSpecName: "kube-api-access-82tbz") pod "24fc2455-9c8e-42eb-a853-125632a7b0bc" (UID: "24fc2455-9c8e-42eb-a853-125632a7b0bc"). InnerVolumeSpecName "kube-api-access-82tbz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 06:56:31.776218 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:31.776186 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24fc2455-9c8e-42eb-a853-125632a7b0bc-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "24fc2455-9c8e-42eb-a853-125632a7b0bc" (UID: "24fc2455-9c8e-42eb-a853-125632a7b0bc"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 06:56:31.842446 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:31.842415 2559 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/24fc2455-9c8e-42eb-a853-125632a7b0bc-postgresdata\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:56:31.842446 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:31.842445 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82tbz\" (UniqueName: \"kubernetes.io/projected/24fc2455-9c8e-42eb-a853-125632a7b0bc-kube-api-access-82tbz\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:56:31.842630 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:31.842457 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mwqvf\" (UniqueName: \"kubernetes.io/projected/855f0fe2-b3f4-4541-b4ce-55c507a3356e-kube-api-access-mwqvf\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:56:32.358629 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:32.358597 2559 generic.go:356] "Generic (PLEG): container finished" podID="855f0fe2-b3f4-4541-b4ce-55c507a3356e" containerID="3f2359588988b12e35a004dcd8bd5ba4fa2d70e9df160570e4dd263154b12f3e" exitCode=0 Sep 09 06:56:32.358774 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:32.358654 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-frltq" Sep 09 06:56:32.358774 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:32.358681 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-frltq" event={"ID":"855f0fe2-b3f4-4541-b4ce-55c507a3356e","Type":"ContainerDied","Data":"3f2359588988b12e35a004dcd8bd5ba4fa2d70e9df160570e4dd263154b12f3e"} Sep 09 06:56:32.358774 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:32.358724 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-frltq" event={"ID":"855f0fe2-b3f4-4541-b4ce-55c507a3356e","Type":"ContainerDied","Data":"7eb2b726f9660066a3f4aba5d70d1df3b744466dec90bbfb4b4a806c91d9721b"} Sep 09 06:56:32.358774 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:32.358744 2559 scope.go:117] "RemoveContainer" containerID="3f2359588988b12e35a004dcd8bd5ba4fa2d70e9df160570e4dd263154b12f3e" Sep 09 06:56:32.361742 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:32.361718 2559 generic.go:356] "Generic (PLEG): container finished" podID="e36dcba8-15e0-4688-8eb1-f465a54feb3d" containerID="2c84cade123a5c9f70ac306dd5d5e16469bdeac0b3468184856d668044d24af6" exitCode=0 Sep 09 06:56:32.361865 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:32.361784 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" event={"ID":"e36dcba8-15e0-4688-8eb1-f465a54feb3d","Type":"ContainerDied","Data":"2c84cade123a5c9f70ac306dd5d5e16469bdeac0b3468184856d668044d24af6"} Sep 09 06:56:32.363662 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:32.363638 2559 generic.go:356] "Generic (PLEG): container finished" podID="24fc2455-9c8e-42eb-a853-125632a7b0bc" containerID="f290dc2b6bd40561f636dc1753d789221afb68e6c726b2e6f66c849a9a241cb2" exitCode=0 Sep 09 06:56:32.363758 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:32.363674 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-xxqvd" event={"ID":"24fc2455-9c8e-42eb-a853-125632a7b0bc","Type":"ContainerDied","Data":"f290dc2b6bd40561f636dc1753d789221afb68e6c726b2e6f66c849a9a241cb2"} Sep 09 06:56:32.363758 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:32.363701 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-xxqvd" Sep 09 06:56:32.363835 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:32.363703 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-xxqvd" event={"ID":"24fc2455-9c8e-42eb-a853-125632a7b0bc","Type":"ContainerDied","Data":"dd763cb2a77415b2dfd7358e7ac789a04b0cc76a0e1c95acd04e47e5e5db3849"} Sep 09 06:56:32.367276 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:32.367204 2559 scope.go:117] "RemoveContainer" containerID="3f2359588988b12e35a004dcd8bd5ba4fa2d70e9df160570e4dd263154b12f3e" Sep 09 06:56:32.367497 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:56:32.367470 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f2359588988b12e35a004dcd8bd5ba4fa2d70e9df160570e4dd263154b12f3e\": container with ID starting with 3f2359588988b12e35a004dcd8bd5ba4fa2d70e9df160570e4dd263154b12f3e not found: ID does not exist" containerID="3f2359588988b12e35a004dcd8bd5ba4fa2d70e9df160570e4dd263154b12f3e" Sep 09 06:56:32.367569 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:32.367506 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f2359588988b12e35a004dcd8bd5ba4fa2d70e9df160570e4dd263154b12f3e"} err="failed to get container status \"3f2359588988b12e35a004dcd8bd5ba4fa2d70e9df160570e4dd263154b12f3e\": rpc error: code = NotFound desc = could not find container \"3f2359588988b12e35a004dcd8bd5ba4fa2d70e9df160570e4dd263154b12f3e\": container with ID starting with 3f2359588988b12e35a004dcd8bd5ba4fa2d70e9df160570e4dd263154b12f3e not found: ID does not exist" Sep 09 06:56:32.367569 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:32.367523 2559 scope.go:117] "RemoveContainer" containerID="f290dc2b6bd40561f636dc1753d789221afb68e6c726b2e6f66c849a9a241cb2" Sep 09 06:56:32.375359 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:32.375342 2559 scope.go:117] "RemoveContainer" containerID="f290dc2b6bd40561f636dc1753d789221afb68e6c726b2e6f66c849a9a241cb2" Sep 09 06:56:32.375641 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:56:32.375623 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f290dc2b6bd40561f636dc1753d789221afb68e6c726b2e6f66c849a9a241cb2\": container with ID starting with f290dc2b6bd40561f636dc1753d789221afb68e6c726b2e6f66c849a9a241cb2 not found: ID does not exist" containerID="f290dc2b6bd40561f636dc1753d789221afb68e6c726b2e6f66c849a9a241cb2" Sep 09 06:56:32.375685 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:32.375649 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f290dc2b6bd40561f636dc1753d789221afb68e6c726b2e6f66c849a9a241cb2"} err="failed to get container status \"f290dc2b6bd40561f636dc1753d789221afb68e6c726b2e6f66c849a9a241cb2\": rpc error: code = NotFound desc = could not find container \"f290dc2b6bd40561f636dc1753d789221afb68e6c726b2e6f66c849a9a241cb2\": container with ID starting with f290dc2b6bd40561f636dc1753d789221afb68e6c726b2e6f66c849a9a241cb2 not found: ID does not exist" Sep 09 06:56:32.380180 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:32.380159 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-frltq"] Sep 09 06:56:32.383690 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:32.383671 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/redis-69bbd798b6-frltq"] Sep 09 06:56:32.394415 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:32.394394 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-xxqvd"] Sep 09 06:56:32.398598 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:32.398576 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/postgres-864846d4f5-xxqvd"] Sep 09 06:56:32.702429 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:32.702390 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24fc2455-9c8e-42eb-a853-125632a7b0bc" path="/var/lib/kubelet/pods/24fc2455-9c8e-42eb-a853-125632a7b0bc/volumes" Sep 09 06:56:32.702929 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:32.702846 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="855f0fe2-b3f4-4541-b4ce-55c507a3356e" path="/var/lib/kubelet/pods/855f0fe2-b3f4-4541-b4ce-55c507a3356e/volumes" Sep 09 06:56:33.031799 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:33.031714 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" podUID="e36dcba8-15e0-4688-8eb1-f465a54feb3d" containerName="online" probeResult="failure" output="Get \"https://10.132.0.35:6567/health\": dial tcp 10.132.0.35:6567: connect: connection refused" Sep 09 06:56:43.032150 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:43.032107 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" podUID="e36dcba8-15e0-4688-8eb1-f465a54feb3d" containerName="online" probeResult="failure" output="Get \"https://10.132.0.35:6567/health\": dial tcp 10.132.0.35:6567: connect: connection refused" Sep 09 06:56:53.031483 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:53.031436 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" podUID="e36dcba8-15e0-4688-8eb1-f465a54feb3d" containerName="online" probeResult="failure" output="Get \"https://10.132.0.35:6567/health\": dial tcp 10.132.0.35:6567: connect: connection refused" Sep 09 06:56:53.031884 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:56:53.031571 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" Sep 09 06:57:01.494533 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:57:01.494500 2559 generic.go:356] "Generic (PLEG): container finished" podID="e36dcba8-15e0-4688-8eb1-f465a54feb3d" containerID="fd22129d13ea7327bddd034364d4d4de20b439c7fff338cb1c3784d1176c06b2" exitCode=137 Sep 09 06:57:01.494533 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:57:01.494526 2559 generic.go:356] "Generic (PLEG): container finished" podID="e36dcba8-15e0-4688-8eb1-f465a54feb3d" containerID="4caeb8088596cab38dca1ec988d29b81434e1aaabc8f8f038aafe20f43bc7c59" exitCode=137 Sep 09 06:57:01.494948 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:57:01.494555 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" event={"ID":"e36dcba8-15e0-4688-8eb1-f465a54feb3d","Type":"ContainerDied","Data":"fd22129d13ea7327bddd034364d4d4de20b439c7fff338cb1c3784d1176c06b2"} Sep 09 06:57:01.494948 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:57:01.494594 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" event={"ID":"e36dcba8-15e0-4688-8eb1-f465a54feb3d","Type":"ContainerDied","Data":"4caeb8088596cab38dca1ec988d29b81434e1aaabc8f8f038aafe20f43bc7c59"} Sep 09 06:57:02.064986 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:57:02.064956 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" Sep 09 06:57:02.191293 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:57:02.191206 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e36dcba8-15e0-4688-8eb1-f465a54feb3d-registry-tls\") pod \"e36dcba8-15e0-4688-8eb1-f465a54feb3d\" (UID: \"e36dcba8-15e0-4688-8eb1-f465a54feb3d\") " Sep 09 06:57:02.191518 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:57:02.191315 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e36dcba8-15e0-4688-8eb1-f465a54feb3d-online-tls\") pod \"e36dcba8-15e0-4688-8eb1-f465a54feb3d\" (UID: \"e36dcba8-15e0-4688-8eb1-f465a54feb3d\") " Sep 09 06:57:02.191518 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:57:02.191402 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e36dcba8-15e0-4688-8eb1-f465a54feb3d-offline-tls\") pod \"e36dcba8-15e0-4688-8eb1-f465a54feb3d\" (UID: \"e36dcba8-15e0-4688-8eb1-f465a54feb3d\") " Sep 09 06:57:02.191518 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:57:02.191447 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kh82m\" (UniqueName: \"kubernetes.io/projected/e36dcba8-15e0-4688-8eb1-f465a54feb3d-kube-api-access-kh82m\") pod \"e36dcba8-15e0-4688-8eb1-f465a54feb3d\" (UID: \"e36dcba8-15e0-4688-8eb1-f465a54feb3d\") " Sep 09 06:57:02.191518 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:57:02.191472 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e36dcba8-15e0-4688-8eb1-f465a54feb3d-feast-data\") pod \"e36dcba8-15e0-4688-8eb1-f465a54feb3d\" (UID: \"e36dcba8-15e0-4688-8eb1-f465a54feb3d\") " Sep 09 06:57:02.193412 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:57:02.193382 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e36dcba8-15e0-4688-8eb1-f465a54feb3d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e36dcba8-15e0-4688-8eb1-f465a54feb3d" (UID: "e36dcba8-15e0-4688-8eb1-f465a54feb3d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 06:57:02.193536 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:57:02.193467 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e36dcba8-15e0-4688-8eb1-f465a54feb3d-online-tls" (OuterVolumeSpecName: "online-tls") pod "e36dcba8-15e0-4688-8eb1-f465a54feb3d" (UID: "e36dcba8-15e0-4688-8eb1-f465a54feb3d"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 06:57:02.193888 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:57:02.193867 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e36dcba8-15e0-4688-8eb1-f465a54feb3d-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "e36dcba8-15e0-4688-8eb1-f465a54feb3d" (UID: "e36dcba8-15e0-4688-8eb1-f465a54feb3d"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 06:57:02.193888 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:57:02.193876 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e36dcba8-15e0-4688-8eb1-f465a54feb3d-kube-api-access-kh82m" (OuterVolumeSpecName: "kube-api-access-kh82m") pod "e36dcba8-15e0-4688-8eb1-f465a54feb3d" (UID: "e36dcba8-15e0-4688-8eb1-f465a54feb3d"). InnerVolumeSpecName "kube-api-access-kh82m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 06:57:02.198904 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:57:02.198874 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e36dcba8-15e0-4688-8eb1-f465a54feb3d-feast-data" (OuterVolumeSpecName: "feast-data") pod "e36dcba8-15e0-4688-8eb1-f465a54feb3d" (UID: "e36dcba8-15e0-4688-8eb1-f465a54feb3d"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 06:57:02.292521 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:57:02.292491 2559 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e36dcba8-15e0-4688-8eb1-f465a54feb3d-online-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:57:02.292521 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:57:02.292517 2559 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e36dcba8-15e0-4688-8eb1-f465a54feb3d-offline-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:57:02.292521 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:57:02.292527 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kh82m\" (UniqueName: \"kubernetes.io/projected/e36dcba8-15e0-4688-8eb1-f465a54feb3d-kube-api-access-kh82m\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:57:02.292721 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:57:02.292537 2559 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e36dcba8-15e0-4688-8eb1-f465a54feb3d-feast-data\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:57:02.292721 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:57:02.292545 2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e36dcba8-15e0-4688-8eb1-f465a54feb3d-registry-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 06:57:02.501598 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:57:02.501510 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" event={"ID":"e36dcba8-15e0-4688-8eb1-f465a54feb3d","Type":"ContainerDied","Data":"547f9b9be1970b188b9aabb048294507aa3818c52483a77f55b15019c80d54ea"} Sep 09 06:57:02.501598 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:57:02.501565 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw" Sep 09 06:57:02.501598 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:57:02.501596 2559 scope.go:117] "RemoveContainer" containerID="fd22129d13ea7327bddd034364d4d4de20b439c7fff338cb1c3784d1176c06b2" Sep 09 06:57:02.510958 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:57:02.510940 2559 scope.go:117] "RemoveContainer" containerID="2c84cade123a5c9f70ac306dd5d5e16469bdeac0b3468184856d668044d24af6" Sep 09 06:57:02.518855 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:57:02.518836 2559 scope.go:117] "RemoveContainer" containerID="4caeb8088596cab38dca1ec988d29b81434e1aaabc8f8f038aafe20f43bc7c59" Sep 09 06:57:02.524449 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:57:02.524427 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw"] Sep 09 06:57:02.527328 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:57:02.527305 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-credit-scoring-7bf7ccdbb-6h9jw"] Sep 09 06:57:02.528251 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:57:02.528043 2559 scope.go:117] "RemoveContainer" containerID="6da4f5bf2aacb9b3cd028d27e7002943c974a4272ae1fe40d093ac623709f508" Sep 09 06:57:02.535434 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:57:02.535415 2559 scope.go:117] "RemoveContainer" containerID="92e8ef2967a6f0918132fda6a5ebe32ca58fcbc158114a8ee0ac651ec5e2049d" Sep 09 06:57:02.701732 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:57:02.701697 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e36dcba8-15e0-4688-8eb1-f465a54feb3d" path="/var/lib/kubelet/pods/e36dcba8-15e0-4688-8eb1-f465a54feb3d/volumes" Sep 09 06:58:22.556605 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:58:22.556581 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nf9gf_26d28788-4f98-454b-ab39-2d0d857aa674/console-operator/2.log" Sep 09 06:58:22.559939 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:58:22.559912 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nf9gf_26d28788-4f98-454b-ab39-2d0d857aa674/console-operator/2.log" Sep 09 06:59:06.264160 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:06.264120 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-nv5p6"] Sep 09 06:59:06.266539 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:06.264647 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e36dcba8-15e0-4688-8eb1-f465a54feb3d" containerName="online" Sep 09 06:59:06.266539 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:06.264663 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e36dcba8-15e0-4688-8eb1-f465a54feb3d" containerName="online" Sep 09 06:59:06.266539 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:06.264678 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e36dcba8-15e0-4688-8eb1-f465a54feb3d" containerName="feast-apply" Sep 09 06:59:06.266539 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:06.264687 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e36dcba8-15e0-4688-8eb1-f465a54feb3d" containerName="feast-apply" Sep 09 06:59:06.266539 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:06.264702 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e36dcba8-15e0-4688-8eb1-f465a54feb3d" containerName="feast-init" Sep 09 06:59:06.266539 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:06.264712 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e36dcba8-15e0-4688-8eb1-f465a54feb3d" containerName="feast-init" Sep 09 06:59:06.266539 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:06.264728 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e36dcba8-15e0-4688-8eb1-f465a54feb3d" containerName="registry" Sep 09 06:59:06.266539 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:06.264738 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e36dcba8-15e0-4688-8eb1-f465a54feb3d" containerName="registry" Sep 09 06:59:06.266539 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:06.264761 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24fc2455-9c8e-42eb-a853-125632a7b0bc" containerName="postgres" Sep 09 06:59:06.266539 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:06.264770 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="24fc2455-9c8e-42eb-a853-125632a7b0bc" containerName="postgres" Sep 09 06:59:06.266539 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:06.264785 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="855f0fe2-b3f4-4541-b4ce-55c507a3356e" containerName="redis" Sep 09 06:59:06.266539 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:06.264793 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="855f0fe2-b3f4-4541-b4ce-55c507a3356e" containerName="redis" Sep 09 06:59:06.266539 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:06.264805 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e36dcba8-15e0-4688-8eb1-f465a54feb3d" containerName="offline" Sep 09 06:59:06.266539 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:06.264813 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e36dcba8-15e0-4688-8eb1-f465a54feb3d" containerName="offline" Sep 09 06:59:06.266539 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:06.264901 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e36dcba8-15e0-4688-8eb1-f465a54feb3d" containerName="registry" Sep 09 06:59:06.266539 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:06.264915 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="855f0fe2-b3f4-4541-b4ce-55c507a3356e" containerName="redis" Sep 09 06:59:06.266539 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:06.264928 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e36dcba8-15e0-4688-8eb1-f465a54feb3d" containerName="offline" Sep 09 06:59:06.266539 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:06.264939 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e36dcba8-15e0-4688-8eb1-f465a54feb3d" containerName="online" Sep 09 06:59:06.266539 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:06.264948 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="24fc2455-9c8e-42eb-a853-125632a7b0bc" containerName="postgres" Sep 09 06:59:06.267994 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:06.267972 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-nv5p6" Sep 09 06:59:06.270432 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:06.270411 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\"" Sep 09 06:59:06.270565 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:06.270542 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-tq2cd\"" Sep 09 06:59:06.271263 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:06.271245 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\"" Sep 09 06:59:06.278137 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:06.278119 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-nv5p6"] Sep 09 06:59:06.389025 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:06.388989 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgx5m\" (UniqueName: \"kubernetes.io/projected/04f835e9-9bba-4530-8ca2-4094714cf65b-kube-api-access-bgx5m\") pod \"redis-69bbd798b6-nv5p6\" (UID: \"04f835e9-9bba-4530-8ca2-4094714cf65b\") " pod="test-ns-feast-rest/redis-69bbd798b6-nv5p6" Sep 09 06:59:06.490203 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:06.490173 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgx5m\" (UniqueName: \"kubernetes.io/projected/04f835e9-9bba-4530-8ca2-4094714cf65b-kube-api-access-bgx5m\") pod \"redis-69bbd798b6-nv5p6\" (UID: \"04f835e9-9bba-4530-8ca2-4094714cf65b\") " pod="test-ns-feast-rest/redis-69bbd798b6-nv5p6" Sep 09 06:59:06.498950 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:06.498918 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgx5m\" (UniqueName: \"kubernetes.io/projected/04f835e9-9bba-4530-8ca2-4094714cf65b-kube-api-access-bgx5m\") pod \"redis-69bbd798b6-nv5p6\" (UID: \"04f835e9-9bba-4530-8ca2-4094714cf65b\") " pod="test-ns-feast-rest/redis-69bbd798b6-nv5p6" Sep 09 06:59:06.584188 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:06.584113 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-nv5p6" Sep 09 06:59:06.852086 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:06.851976 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-nv5p6"] Sep 09 06:59:06.854699 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:59:06.854665 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod04f835e9_9bba_4530_8ca2_4094714cf65b.slice/crio-2ed66c4aa3e153a276d34cacaed34ab2bee57228556a280cebec90ae62f1838a WatchSource:0}: Error finding container 2ed66c4aa3e153a276d34cacaed34ab2bee57228556a280cebec90ae62f1838a: Status 404 returned error can't find the container with id 2ed66c4aa3e153a276d34cacaed34ab2bee57228556a280cebec90ae62f1838a Sep 09 06:59:07.014082 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:07.014029 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-nv5p6" event={"ID":"04f835e9-9bba-4530-8ca2-4094714cf65b","Type":"ContainerStarted","Data":"82556ab7af4d6706e764ca018cd7c69702e7e69c0332a0c71e06cee9dc15f163"} Sep 09 06:59:07.014234 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:07.014090 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-nv5p6" event={"ID":"04f835e9-9bba-4530-8ca2-4094714cf65b","Type":"ContainerStarted","Data":"2ed66c4aa3e153a276d34cacaed34ab2bee57228556a280cebec90ae62f1838a"} Sep 09 06:59:07.027815 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:07.027769 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/redis-69bbd798b6-nv5p6" podStartSLOduration=1.027754 podStartE2EDuration="1.027754s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:59:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:59:07.027185479 +0000 UTC m=+1244.923602902" watchObservedRunningTime="2026-09-09 06:59:07.027754 +0000 UTC m=+1244.924171422" Sep 09 06:59:11.711909 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:11.711875 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-xb5cl"] Sep 09 06:59:11.715278 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:11.715254 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-xb5cl" Sep 09 06:59:11.717776 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:11.717755 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\"" Sep 09 06:59:11.723390 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:11.723353 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-xb5cl"] Sep 09 06:59:11.836526 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:11.836493 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/dfc3cd85-3e6a-4ee2-bee8-c13375ef60eb-postgresdata\") pod \"postgres-864846d4f5-xb5cl\" (UID: \"dfc3cd85-3e6a-4ee2-bee8-c13375ef60eb\") " pod="test-ns-feast-rest/postgres-864846d4f5-xb5cl" Sep 09 06:59:11.836687 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:11.836546 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7lb8\" (UniqueName: \"kubernetes.io/projected/dfc3cd85-3e6a-4ee2-bee8-c13375ef60eb-kube-api-access-k7lb8\") pod \"postgres-864846d4f5-xb5cl\" (UID: \"dfc3cd85-3e6a-4ee2-bee8-c13375ef60eb\") " pod="test-ns-feast-rest/postgres-864846d4f5-xb5cl" Sep 09 06:59:11.937208 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:11.937171 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7lb8\" (UniqueName: \"kubernetes.io/projected/dfc3cd85-3e6a-4ee2-bee8-c13375ef60eb-kube-api-access-k7lb8\") pod \"postgres-864846d4f5-xb5cl\" (UID: \"dfc3cd85-3e6a-4ee2-bee8-c13375ef60eb\") " pod="test-ns-feast-rest/postgres-864846d4f5-xb5cl" Sep 09 06:59:11.937385 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:11.937272 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/dfc3cd85-3e6a-4ee2-bee8-c13375ef60eb-postgresdata\") pod \"postgres-864846d4f5-xb5cl\" (UID: \"dfc3cd85-3e6a-4ee2-bee8-c13375ef60eb\") " pod="test-ns-feast-rest/postgres-864846d4f5-xb5cl" Sep 09 06:59:11.937680 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:11.937661 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/dfc3cd85-3e6a-4ee2-bee8-c13375ef60eb-postgresdata\") pod \"postgres-864846d4f5-xb5cl\" (UID: \"dfc3cd85-3e6a-4ee2-bee8-c13375ef60eb\") " pod="test-ns-feast-rest/postgres-864846d4f5-xb5cl" Sep 09 06:59:11.945816 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:11.945786 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7lb8\" (UniqueName: \"kubernetes.io/projected/dfc3cd85-3e6a-4ee2-bee8-c13375ef60eb-kube-api-access-k7lb8\") pod \"postgres-864846d4f5-xb5cl\" (UID: \"dfc3cd85-3e6a-4ee2-bee8-c13375ef60eb\") " pod="test-ns-feast-rest/postgres-864846d4f5-xb5cl" Sep 09 06:59:12.027180 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:12.027119 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-xb5cl" Sep 09 06:59:12.295592 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:12.295547 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-xb5cl"] Sep 09 06:59:12.298193 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:59:12.298160 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddfc3cd85_3e6a_4ee2_bee8_c13375ef60eb.slice/crio-3ac07683bcef79ebdf8c61bfa40c63033a49ef4ce1ded430bb356dfdc7b05dd2 WatchSource:0}: Error finding container 3ac07683bcef79ebdf8c61bfa40c63033a49ef4ce1ded430bb356dfdc7b05dd2: Status 404 returned error can't find the container with id 3ac07683bcef79ebdf8c61bfa40c63033a49ef4ce1ded430bb356dfdc7b05dd2 Sep 09 06:59:13.041210 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:13.041176 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-xb5cl" event={"ID":"dfc3cd85-3e6a-4ee2-bee8-c13375ef60eb","Type":"ContainerStarted","Data":"8dc3ad3a1ae58441cbf09b952e0ce451e3d971cf980ade7671dfb4a21493ff14"} Sep 09 06:59:13.041210 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:13.041217 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-xb5cl" event={"ID":"dfc3cd85-3e6a-4ee2-bee8-c13375ef60eb","Type":"ContainerStarted","Data":"3ac07683bcef79ebdf8c61bfa40c63033a49ef4ce1ded430bb356dfdc7b05dd2"} Sep 09 06:59:13.057077 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:13.057015 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/postgres-864846d4f5-xb5cl" podStartSLOduration=2.057000031 podStartE2EDuration="2.057000031s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:59:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:59:13.05589954 +0000 UTC m=+1250.952316958" watchObservedRunningTime="2026-09-09 06:59:13.057000031 +0000 UTC m=+1250.953417453" Sep 09 06:59:17.294494 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:17.294460 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm"] Sep 09 06:59:17.298213 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:17.298196 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" Sep 09 06:59:17.300722 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:17.300675 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-registry-tls\"" Sep 09 06:59:17.300722 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:17.300712 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-online-tls\"" Sep 09 06:59:17.300943 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:17.300732 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-dockercfg-xxsb6\"" Sep 09 06:59:17.300943 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:17.300865 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\"" Sep 09 06:59:17.309435 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:17.309412 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm"] Sep 09 06:59:17.382378 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:17.382336 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0ea79a6d-5d15-49c8-8bfc-aa68923db82d-online-tls\") pod \"feast-credit-scoring-6584dcf87c-j78lm\" (UID: \"0ea79a6d-5d15-49c8-8bfc-aa68923db82d\") " pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" Sep 09 06:59:17.382536 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:17.382393 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgzkw\" (UniqueName: \"kubernetes.io/projected/0ea79a6d-5d15-49c8-8bfc-aa68923db82d-kube-api-access-tgzkw\") pod \"feast-credit-scoring-6584dcf87c-j78lm\" (UID: \"0ea79a6d-5d15-49c8-8bfc-aa68923db82d\") " pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" Sep 09 06:59:17.382536 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:17.382501 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0ea79a6d-5d15-49c8-8bfc-aa68923db82d-registry-tls\") pod \"feast-credit-scoring-6584dcf87c-j78lm\" (UID: \"0ea79a6d-5d15-49c8-8bfc-aa68923db82d\") " pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" Sep 09 06:59:17.382536 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:17.382535 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0ea79a6d-5d15-49c8-8bfc-aa68923db82d-feast-data\") pod \"feast-credit-scoring-6584dcf87c-j78lm\" (UID: \"0ea79a6d-5d15-49c8-8bfc-aa68923db82d\") " pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" Sep 09 06:59:17.483794 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:17.483762 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0ea79a6d-5d15-49c8-8bfc-aa68923db82d-registry-tls\") pod \"feast-credit-scoring-6584dcf87c-j78lm\" (UID: \"0ea79a6d-5d15-49c8-8bfc-aa68923db82d\") " pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" Sep 09 06:59:17.483944 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:17.483799 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0ea79a6d-5d15-49c8-8bfc-aa68923db82d-feast-data\") pod \"feast-credit-scoring-6584dcf87c-j78lm\" (UID: \"0ea79a6d-5d15-49c8-8bfc-aa68923db82d\") " pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" Sep 09 06:59:17.483944 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:17.483828 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0ea79a6d-5d15-49c8-8bfc-aa68923db82d-online-tls\") pod \"feast-credit-scoring-6584dcf87c-j78lm\" (UID: \"0ea79a6d-5d15-49c8-8bfc-aa68923db82d\") " pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" Sep 09 06:59:17.483944 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:17.483861 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgzkw\" (UniqueName: \"kubernetes.io/projected/0ea79a6d-5d15-49c8-8bfc-aa68923db82d-kube-api-access-tgzkw\") pod \"feast-credit-scoring-6584dcf87c-j78lm\" (UID: \"0ea79a6d-5d15-49c8-8bfc-aa68923db82d\") " pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" Sep 09 06:59:17.484213 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:17.484193 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0ea79a6d-5d15-49c8-8bfc-aa68923db82d-feast-data\") pod \"feast-credit-scoring-6584dcf87c-j78lm\" (UID: \"0ea79a6d-5d15-49c8-8bfc-aa68923db82d\") " pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" Sep 09 06:59:17.486290 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:17.486266 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0ea79a6d-5d15-49c8-8bfc-aa68923db82d-registry-tls\") pod \"feast-credit-scoring-6584dcf87c-j78lm\" (UID: \"0ea79a6d-5d15-49c8-8bfc-aa68923db82d\") " pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" Sep 09 06:59:17.486409 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:17.486391 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0ea79a6d-5d15-49c8-8bfc-aa68923db82d-online-tls\") pod \"feast-credit-scoring-6584dcf87c-j78lm\" (UID: \"0ea79a6d-5d15-49c8-8bfc-aa68923db82d\") " pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" Sep 09 06:59:17.492835 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:17.492809 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgzkw\" (UniqueName: \"kubernetes.io/projected/0ea79a6d-5d15-49c8-8bfc-aa68923db82d-kube-api-access-tgzkw\") pod \"feast-credit-scoring-6584dcf87c-j78lm\" (UID: \"0ea79a6d-5d15-49c8-8bfc-aa68923db82d\") " pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" Sep 09 06:59:17.608144 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:17.608074 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" Sep 09 06:59:17.885753 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:17.885727 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm"] Sep 09 06:59:17.888425 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:59:17.888397 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ea79a6d_5d15_49c8_8bfc_aa68923db82d.slice/crio-4272054c1fab890be01db44ed40ff881105772b8b2c1997ce4ca532088e2f46f WatchSource:0}: Error finding container 4272054c1fab890be01db44ed40ff881105772b8b2c1997ce4ca532088e2f46f: Status 404 returned error can't find the container with id 4272054c1fab890be01db44ed40ff881105772b8b2c1997ce4ca532088e2f46f Sep 09 06:59:18.067758 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:18.067714 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" event={"ID":"0ea79a6d-5d15-49c8-8bfc-aa68923db82d","Type":"ContainerStarted","Data":"49d60e6006f45271cf791d4e1011ab7dc610c54aab27c3e7a1e7a42b08b0fbb9"} Sep 09 06:59:18.068150 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:18.067810 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" event={"ID":"0ea79a6d-5d15-49c8-8bfc-aa68923db82d","Type":"ContainerStarted","Data":"4272054c1fab890be01db44ed40ff881105772b8b2c1997ce4ca532088e2f46f"} Sep 09 06:59:19.073922 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:19.073884 2559 generic.go:356] "Generic (PLEG): container finished" podID="0ea79a6d-5d15-49c8-8bfc-aa68923db82d" containerID="49d60e6006f45271cf791d4e1011ab7dc610c54aab27c3e7a1e7a42b08b0fbb9" exitCode=0 Sep 09 06:59:19.074388 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:19.073968 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" event={"ID":"0ea79a6d-5d15-49c8-8bfc-aa68923db82d","Type":"ContainerDied","Data":"49d60e6006f45271cf791d4e1011ab7dc610c54aab27c3e7a1e7a42b08b0fbb9"} Sep 09 06:59:20.081610 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:20.081573 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" event={"ID":"0ea79a6d-5d15-49c8-8bfc-aa68923db82d","Type":"ContainerStarted","Data":"37d82fb8f92b79b8985d0d1624d5b231a9b3fc4511c1786d8fa6c123ed6c9a60"} Sep 09 06:59:26.112192 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:26.112158 2559 generic.go:356] "Generic (PLEG): container finished" podID="0ea79a6d-5d15-49c8-8bfc-aa68923db82d" containerID="37d82fb8f92b79b8985d0d1624d5b231a9b3fc4511c1786d8fa6c123ed6c9a60" exitCode=0 Sep 09 06:59:26.112531 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:26.112211 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" event={"ID":"0ea79a6d-5d15-49c8-8bfc-aa68923db82d","Type":"ContainerDied","Data":"37d82fb8f92b79b8985d0d1624d5b231a9b3fc4511c1786d8fa6c123ed6c9a60"} Sep 09 06:59:26.116391 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:26.116369 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\"" Sep 09 06:59:27.119735 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:27.119692 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" event={"ID":"0ea79a6d-5d15-49c8-8bfc-aa68923db82d","Type":"ContainerStarted","Data":"676b1113cc69b3f5e6395133a73bb54c9ed30b477054eb0302b8dd33eb7f8b28"} Sep 09 06:59:27.120158 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:27.119767 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" event={"ID":"0ea79a6d-5d15-49c8-8bfc-aa68923db82d","Type":"ContainerStarted","Data":"c84a0a4a2eed825990e9dbc45e46cf9d1f5fb531bce3e1e84e9bc0055c2e5d9a"} Sep 09 06:59:27.143610 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:27.143549 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" podStartSLOduration=10.143524973 podStartE2EDuration="10.143524973s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:59:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:59:27.139936951 +0000 UTC m=+1265.036354374" watchObservedRunningTime="2026-09-09 06:59:27.143524973 +0000 UTC m=+1265.039942396" Sep 09 06:59:27.608972 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:27.608935 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" Sep 09 06:59:27.609171 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:27.609004 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" Sep 09 06:59:29.608459 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:29.608419 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" Sep 09 06:59:29.608858 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:29.608522 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" Sep 09 06:59:29.610261 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:29.610229 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" podUID="0ea79a6d-5d15-49c8-8bfc-aa68923db82d" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.38:6571: connect: connection refused" Sep 09 06:59:29.610347 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:29.610266 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" podUID="0ea79a6d-5d15-49c8-8bfc-aa68923db82d" containerName="online" probeResult="failure" output="Get \"https://10.132.0.38:6567/health\": dial tcp 10.132.0.38:6567: connect: connection refused" Sep 09 06:59:32.608955 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:32.608913 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" podUID="0ea79a6d-5d15-49c8-8bfc-aa68923db82d" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.38:6571: connect: connection refused" Sep 09 06:59:32.615039 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:32.615012 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" Sep 09 06:59:32.618256 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:32.618230 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" Sep 09 06:59:35.609264 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:35.609220 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" podUID="0ea79a6d-5d15-49c8-8bfc-aa68923db82d" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.38:6571: connect: connection refused" Sep 09 06:59:38.609350 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:38.609311 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" Sep 09 06:59:38.610147 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:38.610124 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" Sep 09 06:59:43.133666 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:43.133633 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp"] Sep 09 06:59:43.139235 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:43.139208 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" Sep 09 06:59:43.141343 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:43.141320 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-registry-tls\"" Sep 09 06:59:43.141439 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:43.141325 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\"" Sep 09 06:59:43.142125 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:43.142110 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-online-tls\"" Sep 09 06:59:43.142176 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:43.142126 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-dockercfg-h7rqs\"" Sep 09 06:59:43.148277 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:43.148255 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp"] Sep 09 06:59:43.204735 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:43.204703 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/85326841-0a68-43aa-b88f-a1a37e02e64d-online-tls\") pod \"feast-driver-ranking-7b99577c94-jwlvp\" (UID: \"85326841-0a68-43aa-b88f-a1a37e02e64d\") " pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" Sep 09 06:59:43.204924 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:43.204749 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmlv8\" (UniqueName: \"kubernetes.io/projected/85326841-0a68-43aa-b88f-a1a37e02e64d-kube-api-access-jmlv8\") pod \"feast-driver-ranking-7b99577c94-jwlvp\" (UID: \"85326841-0a68-43aa-b88f-a1a37e02e64d\") " pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" Sep 09 06:59:43.204924 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:43.204773 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/85326841-0a68-43aa-b88f-a1a37e02e64d-registry-tls\") pod \"feast-driver-ranking-7b99577c94-jwlvp\" (UID: \"85326841-0a68-43aa-b88f-a1a37e02e64d\") " pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" Sep 09 06:59:43.205094 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:43.204954 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/85326841-0a68-43aa-b88f-a1a37e02e64d-feast-data\") pod \"feast-driver-ranking-7b99577c94-jwlvp\" (UID: \"85326841-0a68-43aa-b88f-a1a37e02e64d\") " pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" Sep 09 06:59:43.305516 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:43.305484 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmlv8\" (UniqueName: \"kubernetes.io/projected/85326841-0a68-43aa-b88f-a1a37e02e64d-kube-api-access-jmlv8\") pod \"feast-driver-ranking-7b99577c94-jwlvp\" (UID: \"85326841-0a68-43aa-b88f-a1a37e02e64d\") " pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" Sep 09 06:59:43.305516 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:43.305518 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/85326841-0a68-43aa-b88f-a1a37e02e64d-registry-tls\") pod \"feast-driver-ranking-7b99577c94-jwlvp\" (UID: \"85326841-0a68-43aa-b88f-a1a37e02e64d\") " pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" Sep 09 06:59:43.305741 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:43.305551 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/85326841-0a68-43aa-b88f-a1a37e02e64d-feast-data\") pod \"feast-driver-ranking-7b99577c94-jwlvp\" (UID: \"85326841-0a68-43aa-b88f-a1a37e02e64d\") " pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" Sep 09 06:59:43.305741 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:43.305583 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/85326841-0a68-43aa-b88f-a1a37e02e64d-online-tls\") pod \"feast-driver-ranking-7b99577c94-jwlvp\" (UID: \"85326841-0a68-43aa-b88f-a1a37e02e64d\") " pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" Sep 09 06:59:43.305741 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:59:43.305648 2559 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-driver-ranking-registry-tls: secret "feast-driver-ranking-registry-tls" not found Sep 09 06:59:43.305741 ip-10-0-132-137 kubenswrapper[2559]: E0909 06:59:43.305731 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85326841-0a68-43aa-b88f-a1a37e02e64d-registry-tls podName:85326841-0a68-43aa-b88f-a1a37e02e64d nodeName:}" failed. No retries permitted until 2026-09-09 06:59:43.805706653 +0000 UTC m=+1281.702124072 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/85326841-0a68-43aa-b88f-a1a37e02e64d-registry-tls") pod "feast-driver-ranking-7b99577c94-jwlvp" (UID: "85326841-0a68-43aa-b88f-a1a37e02e64d") : secret "feast-driver-ranking-registry-tls" not found Sep 09 06:59:43.305982 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:43.305963 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/85326841-0a68-43aa-b88f-a1a37e02e64d-feast-data\") pod \"feast-driver-ranking-7b99577c94-jwlvp\" (UID: \"85326841-0a68-43aa-b88f-a1a37e02e64d\") " pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" Sep 09 06:59:43.307905 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:43.307890 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/85326841-0a68-43aa-b88f-a1a37e02e64d-online-tls\") pod \"feast-driver-ranking-7b99577c94-jwlvp\" (UID: \"85326841-0a68-43aa-b88f-a1a37e02e64d\") " pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" Sep 09 06:59:43.313374 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:43.313353 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmlv8\" (UniqueName: \"kubernetes.io/projected/85326841-0a68-43aa-b88f-a1a37e02e64d-kube-api-access-jmlv8\") pod \"feast-driver-ranking-7b99577c94-jwlvp\" (UID: \"85326841-0a68-43aa-b88f-a1a37e02e64d\") " pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" Sep 09 06:59:43.810486 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:43.810450 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/85326841-0a68-43aa-b88f-a1a37e02e64d-registry-tls\") pod \"feast-driver-ranking-7b99577c94-jwlvp\" (UID: \"85326841-0a68-43aa-b88f-a1a37e02e64d\") " pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" Sep 09 06:59:43.812780 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:43.812756 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/85326841-0a68-43aa-b88f-a1a37e02e64d-registry-tls\") pod \"feast-driver-ranking-7b99577c94-jwlvp\" (UID: \"85326841-0a68-43aa-b88f-a1a37e02e64d\") " pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" Sep 09 06:59:44.049736 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:44.049698 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" Sep 09 06:59:44.327151 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:44.327129 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp"] Sep 09 06:59:44.328908 ip-10-0-132-137 kubenswrapper[2559]: W0909 06:59:44.328872 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85326841_0a68_43aa_b88f_a1a37e02e64d.slice/crio-45676b97efcfad2421a245b9066c6bae3ca88a13950231f04b25639632c22419 WatchSource:0}: Error finding container 45676b97efcfad2421a245b9066c6bae3ca88a13950231f04b25639632c22419: Status 404 returned error can't find the container with id 45676b97efcfad2421a245b9066c6bae3ca88a13950231f04b25639632c22419 Sep 09 06:59:45.213667 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:45.213632 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" event={"ID":"85326841-0a68-43aa-b88f-a1a37e02e64d","Type":"ContainerStarted","Data":"16ec7b10d7ba9084ed703eba4b9a5cb36aa2f72a963fbc4168a091672ae0dac7"} Sep 09 06:59:45.213831 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:45.213686 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" event={"ID":"85326841-0a68-43aa-b88f-a1a37e02e64d","Type":"ContainerStarted","Data":"45676b97efcfad2421a245b9066c6bae3ca88a13950231f04b25639632c22419"} Sep 09 06:59:49.234199 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:49.234166 2559 generic.go:356] "Generic (PLEG): container finished" podID="85326841-0a68-43aa-b88f-a1a37e02e64d" containerID="16ec7b10d7ba9084ed703eba4b9a5cb36aa2f72a963fbc4168a091672ae0dac7" exitCode=0 Sep 09 06:59:49.234574 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:49.234245 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" event={"ID":"85326841-0a68-43aa-b88f-a1a37e02e64d","Type":"ContainerDied","Data":"16ec7b10d7ba9084ed703eba4b9a5cb36aa2f72a963fbc4168a091672ae0dac7"} Sep 09 06:59:50.241447 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:50.241412 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" event={"ID":"85326841-0a68-43aa-b88f-a1a37e02e64d","Type":"ContainerStarted","Data":"a6634df0650f0679bb5a23532bd5a714c4c71365288aad1c5336d3b4ca7f930d"} Sep 09 06:59:56.274850 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:56.274760 2559 generic.go:356] "Generic (PLEG): container finished" podID="85326841-0a68-43aa-b88f-a1a37e02e64d" containerID="a6634df0650f0679bb5a23532bd5a714c4c71365288aad1c5336d3b4ca7f930d" exitCode=0 Sep 09 06:59:56.274850 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:56.274831 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" event={"ID":"85326841-0a68-43aa-b88f-a1a37e02e64d","Type":"ContainerDied","Data":"a6634df0650f0679bb5a23532bd5a714c4c71365288aad1c5336d3b4ca7f930d"} Sep 09 06:59:57.282703 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:57.282662 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" event={"ID":"85326841-0a68-43aa-b88f-a1a37e02e64d","Type":"ContainerStarted","Data":"595340ef7c96890b83629f8b5b02d98c21232abc316ee4388f5054982f90aaee"} Sep 09 06:59:57.283221 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:57.282728 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" event={"ID":"85326841-0a68-43aa-b88f-a1a37e02e64d","Type":"ContainerStarted","Data":"1e676fe8599a563e7dca41efa81e3c17d4ba833663792b4203e92c80a29f756d"} Sep 09 06:59:57.302952 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:57.302897 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" podStartSLOduration=14.302876109 podStartE2EDuration="14.302876109s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:59:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:59:57.301515424 +0000 UTC m=+1295.197932845" watchObservedRunningTime="2026-09-09 06:59:57.302876109 +0000 UTC m=+1295.199293532" Sep 09 06:59:59.050043 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:59.050001 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" Sep 09 06:59:59.050563 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:59.050091 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" Sep 09 06:59:59.051787 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:59.051758 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" podUID="85326841-0a68-43aa-b88f-a1a37e02e64d" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.39:6571: connect: connection refused" Sep 09 06:59:59.051931 ip-10-0-132-137 kubenswrapper[2559]: I0909 06:59:59.051805 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" podUID="85326841-0a68-43aa-b88f-a1a37e02e64d" containerName="online" probeResult="failure" output="Get \"https://10.132.0.39:6567/health\": dial tcp 10.132.0.39:6567: connect: connection refused" Sep 09 07:00:02.050936 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:00:02.050891 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" podUID="85326841-0a68-43aa-b88f-a1a37e02e64d" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.39:6571: connect: connection refused" Sep 09 07:00:02.051429 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:00:02.051020 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" podUID="85326841-0a68-43aa-b88f-a1a37e02e64d" containerName="online" probeResult="failure" output="Get \"https://10.132.0.39:6567/health\": dial tcp 10.132.0.39:6567: connect: connection refused" Sep 09 07:00:04.050063 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:00:04.050010 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" Sep 09 07:00:04.050665 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:00:04.050089 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" Sep 09 07:00:05.051072 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:00:05.051004 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" podUID="85326841-0a68-43aa-b88f-a1a37e02e64d" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.39:6571: connect: connection refused" Sep 09 07:00:05.059530 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:00:05.059511 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" Sep 09 07:00:05.063797 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:00:05.063767 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" Sep 09 07:00:08.049983 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:00:08.049939 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" podUID="85326841-0a68-43aa-b88f-a1a37e02e64d" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.39:6571: connect: connection refused" Sep 09 07:00:11.051314 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:00:11.051277 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" Sep 09 07:00:11.052153 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:00:11.052130 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" Sep 09 07:01:30.783572 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:30.783539 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-98b9c5856-7bk87"] Sep 09 07:01:30.784157 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:30.783736 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-98b9c5856-7bk87" podUID="247e9f69-e45c-49e7-be80-42c2ad4e0e0f" containerName="manager" containerID="cri-o://324c40df5415380b5e02c3e967557dc90d655802556450aa57e322ec753afaa8" gracePeriod=10 Sep 09 07:01:31.030155 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:31.030037 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm"] Sep 09 07:01:31.030406 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:31.030351 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" podUID="0ea79a6d-5d15-49c8-8bfc-aa68923db82d" containerName="registry" containerID="cri-o://c84a0a4a2eed825990e9dbc45e46cf9d1f5fb531bce3e1e84e9bc0055c2e5d9a" gracePeriod=30 Sep 09 07:01:31.030712 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:31.030409 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" podUID="0ea79a6d-5d15-49c8-8bfc-aa68923db82d" containerName="online" containerID="cri-o://676b1113cc69b3f5e6395133a73bb54c9ed30b477054eb0302b8dd33eb7f8b28" gracePeriod=30 Sep 09 07:01:31.034180 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:31.034116 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp"] Sep 09 07:01:31.034395 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:31.034346 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" podUID="85326841-0a68-43aa-b88f-a1a37e02e64d" containerName="registry" containerID="cri-o://1e676fe8599a563e7dca41efa81e3c17d4ba833663792b4203e92c80a29f756d" gracePeriod=30 Sep 09 07:01:31.034687 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:31.034539 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" podUID="85326841-0a68-43aa-b88f-a1a37e02e64d" containerName="online" containerID="cri-o://595340ef7c96890b83629f8b5b02d98c21232abc316ee4388f5054982f90aaee" gracePeriod=30 Sep 09 07:01:31.070382 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:31.070355 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-98b9c5856-7bk87" Sep 09 07:01:31.126134 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:31.126098 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ssz2k\" (UniqueName: \"kubernetes.io/projected/247e9f69-e45c-49e7-be80-42c2ad4e0e0f-kube-api-access-ssz2k\") pod \"247e9f69-e45c-49e7-be80-42c2ad4e0e0f\" (UID: \"247e9f69-e45c-49e7-be80-42c2ad4e0e0f\") " Sep 09 07:01:31.128236 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:31.128205 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/247e9f69-e45c-49e7-be80-42c2ad4e0e0f-kube-api-access-ssz2k" (OuterVolumeSpecName: "kube-api-access-ssz2k") pod "247e9f69-e45c-49e7-be80-42c2ad4e0e0f" (UID: "247e9f69-e45c-49e7-be80-42c2ad4e0e0f"). InnerVolumeSpecName "kube-api-access-ssz2k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 07:01:31.227018 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:31.226975 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ssz2k\" (UniqueName: \"kubernetes.io/projected/247e9f69-e45c-49e7-be80-42c2ad4e0e0f-kube-api-access-ssz2k\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:01:31.758329 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:31.758289 2559 generic.go:356] "Generic (PLEG): container finished" podID="247e9f69-e45c-49e7-be80-42c2ad4e0e0f" containerID="324c40df5415380b5e02c3e967557dc90d655802556450aa57e322ec753afaa8" exitCode=0 Sep 09 07:01:31.758486 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:31.758349 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-98b9c5856-7bk87" Sep 09 07:01:31.758486 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:31.758378 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-98b9c5856-7bk87" event={"ID":"247e9f69-e45c-49e7-be80-42c2ad4e0e0f","Type":"ContainerDied","Data":"324c40df5415380b5e02c3e967557dc90d655802556450aa57e322ec753afaa8"} Sep 09 07:01:31.758486 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:31.758437 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-98b9c5856-7bk87" event={"ID":"247e9f69-e45c-49e7-be80-42c2ad4e0e0f","Type":"ContainerDied","Data":"cea0fa9cb7d11c678902b7bde86a312362d0e6c38e83c8db2c302b66d3c2b0cf"} Sep 09 07:01:31.758486 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:31.758461 2559 scope.go:117] "RemoveContainer" containerID="324c40df5415380b5e02c3e967557dc90d655802556450aa57e322ec753afaa8" Sep 09 07:01:31.767857 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:31.767819 2559 scope.go:117] "RemoveContainer" containerID="324c40df5415380b5e02c3e967557dc90d655802556450aa57e322ec753afaa8" Sep 09 07:01:31.768318 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:01:31.768288 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"324c40df5415380b5e02c3e967557dc90d655802556450aa57e322ec753afaa8\": container with ID starting with 324c40df5415380b5e02c3e967557dc90d655802556450aa57e322ec753afaa8 not found: ID does not exist" containerID="324c40df5415380b5e02c3e967557dc90d655802556450aa57e322ec753afaa8" Sep 09 07:01:31.768434 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:31.768325 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"324c40df5415380b5e02c3e967557dc90d655802556450aa57e322ec753afaa8"} err="failed to get container status \"324c40df5415380b5e02c3e967557dc90d655802556450aa57e322ec753afaa8\": rpc error: code = NotFound desc = could not find container \"324c40df5415380b5e02c3e967557dc90d655802556450aa57e322ec753afaa8\": container with ID starting with 324c40df5415380b5e02c3e967557dc90d655802556450aa57e322ec753afaa8 not found: ID does not exist" Sep 09 07:01:31.781799 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:31.781761 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-98b9c5856-7bk87"] Sep 09 07:01:31.783390 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:31.783362 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-98b9c5856-7bk87"] Sep 09 07:01:32.615887 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:32.615845 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" podUID="0ea79a6d-5d15-49c8-8bfc-aa68923db82d" containerName="online" probeResult="failure" output="Get \"https://10.132.0.38:6567/health\": dial tcp 10.132.0.38:6567: connect: connection refused" Sep 09 07:01:32.707546 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:32.707513 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="247e9f69-e45c-49e7-be80-42c2ad4e0e0f" path="/var/lib/kubelet/pods/247e9f69-e45c-49e7-be80-42c2ad4e0e0f/volumes" Sep 09 07:01:32.765975 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:32.765944 2559 generic.go:356] "Generic (PLEG): container finished" podID="85326841-0a68-43aa-b88f-a1a37e02e64d" containerID="595340ef7c96890b83629f8b5b02d98c21232abc316ee4388f5054982f90aaee" exitCode=0 Sep 09 07:01:32.766152 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:32.766016 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" event={"ID":"85326841-0a68-43aa-b88f-a1a37e02e64d","Type":"ContainerDied","Data":"595340ef7c96890b83629f8b5b02d98c21232abc316ee4388f5054982f90aaee"} Sep 09 07:01:32.768958 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:32.768936 2559 generic.go:356] "Generic (PLEG): container finished" podID="0ea79a6d-5d15-49c8-8bfc-aa68923db82d" containerID="676b1113cc69b3f5e6395133a73bb54c9ed30b477054eb0302b8dd33eb7f8b28" exitCode=0 Sep 09 07:01:32.769079 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:32.768965 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" event={"ID":"0ea79a6d-5d15-49c8-8bfc-aa68923db82d","Type":"ContainerDied","Data":"676b1113cc69b3f5e6395133a73bb54c9ed30b477054eb0302b8dd33eb7f8b28"} Sep 09 07:01:33.528071 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:33.528025 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-xb5cl"] Sep 09 07:01:33.528237 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:33.528209 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/postgres-864846d4f5-xb5cl" podUID="dfc3cd85-3e6a-4ee2-bee8-c13375ef60eb" containerName="postgres" containerID="cri-o://8dc3ad3a1ae58441cbf09b952e0ce451e3d971cf980ade7671dfb4a21493ff14" gracePeriod=30 Sep 09 07:01:33.532625 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:33.532601 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-nv5p6"] Sep 09 07:01:33.532765 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:33.532746 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/redis-69bbd798b6-nv5p6" podUID="04f835e9-9bba-4530-8ca2-4094714cf65b" containerName="redis" containerID="cri-o://82556ab7af4d6706e764ca018cd7c69702e7e69c0332a0c71e06cee9dc15f163" gracePeriod=30 Sep 09 07:01:33.776935 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:33.776902 2559 generic.go:356] "Generic (PLEG): container finished" podID="dfc3cd85-3e6a-4ee2-bee8-c13375ef60eb" containerID="8dc3ad3a1ae58441cbf09b952e0ce451e3d971cf980ade7671dfb4a21493ff14" exitCode=0 Sep 09 07:01:33.777340 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:33.777017 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-xb5cl" event={"ID":"dfc3cd85-3e6a-4ee2-bee8-c13375ef60eb","Type":"ContainerDied","Data":"8dc3ad3a1ae58441cbf09b952e0ce451e3d971cf980ade7671dfb4a21493ff14"} Sep 09 07:01:33.778538 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:33.778487 2559 generic.go:356] "Generic (PLEG): container finished" podID="04f835e9-9bba-4530-8ca2-4094714cf65b" containerID="82556ab7af4d6706e764ca018cd7c69702e7e69c0332a0c71e06cee9dc15f163" exitCode=0 Sep 09 07:01:33.778633 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:33.778535 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-nv5p6" event={"ID":"04f835e9-9bba-4530-8ca2-4094714cf65b","Type":"ContainerDied","Data":"82556ab7af4d6706e764ca018cd7c69702e7e69c0332a0c71e06cee9dc15f163"} Sep 09 07:01:33.795504 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:33.795482 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-nv5p6" Sep 09 07:01:33.798547 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:33.798531 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-xb5cl" Sep 09 07:01:33.947122 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:33.947085 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bgx5m\" (UniqueName: \"kubernetes.io/projected/04f835e9-9bba-4530-8ca2-4094714cf65b-kube-api-access-bgx5m\") pod \"04f835e9-9bba-4530-8ca2-4094714cf65b\" (UID: \"04f835e9-9bba-4530-8ca2-4094714cf65b\") " Sep 09 07:01:33.947122 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:33.947126 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7lb8\" (UniqueName: \"kubernetes.io/projected/dfc3cd85-3e6a-4ee2-bee8-c13375ef60eb-kube-api-access-k7lb8\") pod \"dfc3cd85-3e6a-4ee2-bee8-c13375ef60eb\" (UID: \"dfc3cd85-3e6a-4ee2-bee8-c13375ef60eb\") " Sep 09 07:01:33.947329 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:33.947191 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/dfc3cd85-3e6a-4ee2-bee8-c13375ef60eb-postgresdata\") pod \"dfc3cd85-3e6a-4ee2-bee8-c13375ef60eb\" (UID: \"dfc3cd85-3e6a-4ee2-bee8-c13375ef60eb\") " Sep 09 07:01:33.949318 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:33.949271 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfc3cd85-3e6a-4ee2-bee8-c13375ef60eb-kube-api-access-k7lb8" (OuterVolumeSpecName: "kube-api-access-k7lb8") pod "dfc3cd85-3e6a-4ee2-bee8-c13375ef60eb" (UID: "dfc3cd85-3e6a-4ee2-bee8-c13375ef60eb"). InnerVolumeSpecName "kube-api-access-k7lb8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 07:01:33.949451 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:33.949328 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04f835e9-9bba-4530-8ca2-4094714cf65b-kube-api-access-bgx5m" (OuterVolumeSpecName: "kube-api-access-bgx5m") pod "04f835e9-9bba-4530-8ca2-4094714cf65b" (UID: "04f835e9-9bba-4530-8ca2-4094714cf65b"). InnerVolumeSpecName "kube-api-access-bgx5m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 07:01:33.977280 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:33.977250 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfc3cd85-3e6a-4ee2-bee8-c13375ef60eb-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "dfc3cd85-3e6a-4ee2-bee8-c13375ef60eb" (UID: "dfc3cd85-3e6a-4ee2-bee8-c13375ef60eb"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 07:01:34.048394 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:34.048322 2559 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/dfc3cd85-3e6a-4ee2-bee8-c13375ef60eb-postgresdata\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:01:34.048394 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:34.048352 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgx5m\" (UniqueName: \"kubernetes.io/projected/04f835e9-9bba-4530-8ca2-4094714cf65b-kube-api-access-bgx5m\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:01:34.048394 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:34.048363 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7lb8\" (UniqueName: \"kubernetes.io/projected/dfc3cd85-3e6a-4ee2-bee8-c13375ef60eb-kube-api-access-k7lb8\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:01:34.784182 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:34.784151 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-xb5cl" Sep 09 07:01:34.784581 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:34.784198 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-xb5cl" event={"ID":"dfc3cd85-3e6a-4ee2-bee8-c13375ef60eb","Type":"ContainerDied","Data":"3ac07683bcef79ebdf8c61bfa40c63033a49ef4ce1ded430bb356dfdc7b05dd2"} Sep 09 07:01:34.784581 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:34.784244 2559 scope.go:117] "RemoveContainer" containerID="8dc3ad3a1ae58441cbf09b952e0ce451e3d971cf980ade7671dfb4a21493ff14" Sep 09 07:01:34.785598 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:34.785573 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-nv5p6" event={"ID":"04f835e9-9bba-4530-8ca2-4094714cf65b","Type":"ContainerDied","Data":"2ed66c4aa3e153a276d34cacaed34ab2bee57228556a280cebec90ae62f1838a"} Sep 09 07:01:34.785684 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:34.785587 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-nv5p6" Sep 09 07:01:34.792716 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:34.792698 2559 scope.go:117] "RemoveContainer" containerID="82556ab7af4d6706e764ca018cd7c69702e7e69c0332a0c71e06cee9dc15f163" Sep 09 07:01:34.802404 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:34.802385 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-xb5cl"] Sep 09 07:01:34.806398 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:34.806378 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-xb5cl"] Sep 09 07:01:34.815766 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:34.815744 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-nv5p6"] Sep 09 07:01:34.820959 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:34.820938 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-nv5p6"] Sep 09 07:01:35.060502 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:35.060416 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" podUID="85326841-0a68-43aa-b88f-a1a37e02e64d" containerName="online" probeResult="failure" output="Get \"https://10.132.0.39:6567/health\": dial tcp 10.132.0.39:6567: connect: connection refused" Sep 09 07:01:36.707532 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:36.707495 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04f835e9-9bba-4530-8ca2-4094714cf65b" path="/var/lib/kubelet/pods/04f835e9-9bba-4530-8ca2-4094714cf65b/volumes" Sep 09 07:01:36.708009 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:36.707826 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfc3cd85-3e6a-4ee2-bee8-c13375ef60eb" path="/var/lib/kubelet/pods/dfc3cd85-3e6a-4ee2-bee8-c13375ef60eb/volumes" Sep 09 07:01:42.616401 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:42.616359 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" podUID="0ea79a6d-5d15-49c8-8bfc-aa68923db82d" containerName="online" probeResult="failure" output="Get \"https://10.132.0.38:6567/health\": dial tcp 10.132.0.38:6567: connect: connection refused" Sep 09 07:01:45.060250 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:45.060209 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" podUID="85326841-0a68-43aa-b88f-a1a37e02e64d" containerName="online" probeResult="failure" output="Get \"https://10.132.0.39:6567/health\": dial tcp 10.132.0.39:6567: connect: connection refused" Sep 09 07:01:52.615743 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:52.615696 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" podUID="0ea79a6d-5d15-49c8-8bfc-aa68923db82d" containerName="online" probeResult="failure" output="Get \"https://10.132.0.38:6567/health\": dial tcp 10.132.0.38:6567: connect: connection refused" Sep 09 07:01:52.616146 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:52.615848 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" Sep 09 07:01:55.060793 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:55.060752 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" podUID="85326841-0a68-43aa-b88f-a1a37e02e64d" containerName="online" probeResult="failure" output="Get \"https://10.132.0.39:6567/health\": dial tcp 10.132.0.39:6567: connect: connection refused" Sep 09 07:01:55.061222 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:01:55.060863 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" Sep 09 07:02:01.702739 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.702714 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" Sep 09 07:02:01.705176 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.705158 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" Sep 09 07:02:01.778829 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.778795 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0ea79a6d-5d15-49c8-8bfc-aa68923db82d-feast-data\") pod \"0ea79a6d-5d15-49c8-8bfc-aa68923db82d\" (UID: \"0ea79a6d-5d15-49c8-8bfc-aa68923db82d\") " Sep 09 07:02:01.779008 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.778844 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/85326841-0a68-43aa-b88f-a1a37e02e64d-online-tls\") pod \"85326841-0a68-43aa-b88f-a1a37e02e64d\" (UID: \"85326841-0a68-43aa-b88f-a1a37e02e64d\") " Sep 09 07:02:01.779008 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.778871 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/85326841-0a68-43aa-b88f-a1a37e02e64d-registry-tls\") pod \"85326841-0a68-43aa-b88f-a1a37e02e64d\" (UID: \"85326841-0a68-43aa-b88f-a1a37e02e64d\") " Sep 09 07:02:01.779008 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.778887 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/85326841-0a68-43aa-b88f-a1a37e02e64d-feast-data\") pod \"85326841-0a68-43aa-b88f-a1a37e02e64d\" (UID: \"85326841-0a68-43aa-b88f-a1a37e02e64d\") " Sep 09 07:02:01.779008 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.778939 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0ea79a6d-5d15-49c8-8bfc-aa68923db82d-online-tls\") pod \"0ea79a6d-5d15-49c8-8bfc-aa68923db82d\" (UID: \"0ea79a6d-5d15-49c8-8bfc-aa68923db82d\") " Sep 09 07:02:01.779008 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.778991 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmlv8\" (UniqueName: \"kubernetes.io/projected/85326841-0a68-43aa-b88f-a1a37e02e64d-kube-api-access-jmlv8\") pod \"85326841-0a68-43aa-b88f-a1a37e02e64d\" (UID: \"85326841-0a68-43aa-b88f-a1a37e02e64d\") " Sep 09 07:02:01.779313 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.779017 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tgzkw\" (UniqueName: \"kubernetes.io/projected/0ea79a6d-5d15-49c8-8bfc-aa68923db82d-kube-api-access-tgzkw\") pod \"0ea79a6d-5d15-49c8-8bfc-aa68923db82d\" (UID: \"0ea79a6d-5d15-49c8-8bfc-aa68923db82d\") " Sep 09 07:02:01.779313 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.779039 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0ea79a6d-5d15-49c8-8bfc-aa68923db82d-registry-tls\") pod \"0ea79a6d-5d15-49c8-8bfc-aa68923db82d\" (UID: \"0ea79a6d-5d15-49c8-8bfc-aa68923db82d\") " Sep 09 07:02:01.779714 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.779459 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85326841-0a68-43aa-b88f-a1a37e02e64d-feast-data" (OuterVolumeSpecName: "feast-data") pod "85326841-0a68-43aa-b88f-a1a37e02e64d" (UID: "85326841-0a68-43aa-b88f-a1a37e02e64d"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 07:02:01.781709 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.781676 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85326841-0a68-43aa-b88f-a1a37e02e64d-online-tls" (OuterVolumeSpecName: "online-tls") pod "85326841-0a68-43aa-b88f-a1a37e02e64d" (UID: "85326841-0a68-43aa-b88f-a1a37e02e64d"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 07:02:01.781820 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.781720 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ea79a6d-5d15-49c8-8bfc-aa68923db82d-online-tls" (OuterVolumeSpecName: "online-tls") pod "0ea79a6d-5d15-49c8-8bfc-aa68923db82d" (UID: "0ea79a6d-5d15-49c8-8bfc-aa68923db82d"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 07:02:01.781820 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.781768 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ea79a6d-5d15-49c8-8bfc-aa68923db82d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "0ea79a6d-5d15-49c8-8bfc-aa68923db82d" (UID: "0ea79a6d-5d15-49c8-8bfc-aa68923db82d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 07:02:01.781820 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.781782 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85326841-0a68-43aa-b88f-a1a37e02e64d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "85326841-0a68-43aa-b88f-a1a37e02e64d" (UID: "85326841-0a68-43aa-b88f-a1a37e02e64d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 07:02:01.781944 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.781875 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85326841-0a68-43aa-b88f-a1a37e02e64d-kube-api-access-jmlv8" (OuterVolumeSpecName: "kube-api-access-jmlv8") pod "85326841-0a68-43aa-b88f-a1a37e02e64d" (UID: "85326841-0a68-43aa-b88f-a1a37e02e64d"). InnerVolumeSpecName "kube-api-access-jmlv8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 07:02:01.781944 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.781887 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ea79a6d-5d15-49c8-8bfc-aa68923db82d-kube-api-access-tgzkw" (OuterVolumeSpecName: "kube-api-access-tgzkw") pod "0ea79a6d-5d15-49c8-8bfc-aa68923db82d" (UID: "0ea79a6d-5d15-49c8-8bfc-aa68923db82d"). InnerVolumeSpecName "kube-api-access-tgzkw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 07:02:01.788288 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.788267 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ea79a6d-5d15-49c8-8bfc-aa68923db82d-feast-data" (OuterVolumeSpecName: "feast-data") pod "0ea79a6d-5d15-49c8-8bfc-aa68923db82d" (UID: "0ea79a6d-5d15-49c8-8bfc-aa68923db82d"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 07:02:01.880155 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.880045 2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/85326841-0a68-43aa-b88f-a1a37e02e64d-registry-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:02:01.880155 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.880106 2559 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/85326841-0a68-43aa-b88f-a1a37e02e64d-feast-data\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:02:01.880155 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.880115 2559 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0ea79a6d-5d15-49c8-8bfc-aa68923db82d-online-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:02:01.880155 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.880124 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmlv8\" (UniqueName: \"kubernetes.io/projected/85326841-0a68-43aa-b88f-a1a37e02e64d-kube-api-access-jmlv8\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:02:01.880155 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.880133 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tgzkw\" (UniqueName: \"kubernetes.io/projected/0ea79a6d-5d15-49c8-8bfc-aa68923db82d-kube-api-access-tgzkw\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:02:01.880155 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.880141 2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0ea79a6d-5d15-49c8-8bfc-aa68923db82d-registry-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:02:01.880155 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.880149 2559 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0ea79a6d-5d15-49c8-8bfc-aa68923db82d-feast-data\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:02:01.880155 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.880156 2559 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/85326841-0a68-43aa-b88f-a1a37e02e64d-online-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:02:01.906411 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.906375 2559 generic.go:356] "Generic (PLEG): container finished" podID="0ea79a6d-5d15-49c8-8bfc-aa68923db82d" containerID="c84a0a4a2eed825990e9dbc45e46cf9d1f5fb531bce3e1e84e9bc0055c2e5d9a" exitCode=137 Sep 09 07:02:01.906526 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.906442 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" Sep 09 07:02:01.906526 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.906460 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" event={"ID":"0ea79a6d-5d15-49c8-8bfc-aa68923db82d","Type":"ContainerDied","Data":"c84a0a4a2eed825990e9dbc45e46cf9d1f5fb531bce3e1e84e9bc0055c2e5d9a"} Sep 09 07:02:01.906526 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.906513 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm" event={"ID":"0ea79a6d-5d15-49c8-8bfc-aa68923db82d","Type":"ContainerDied","Data":"4272054c1fab890be01db44ed40ff881105772b8b2c1997ce4ca532088e2f46f"} Sep 09 07:02:01.906667 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.906528 2559 scope.go:117] "RemoveContainer" containerID="676b1113cc69b3f5e6395133a73bb54c9ed30b477054eb0302b8dd33eb7f8b28" Sep 09 07:02:01.908861 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.908842 2559 generic.go:356] "Generic (PLEG): container finished" podID="85326841-0a68-43aa-b88f-a1a37e02e64d" containerID="1e676fe8599a563e7dca41efa81e3c17d4ba833663792b4203e92c80a29f756d" exitCode=137 Sep 09 07:02:01.908949 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.908900 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" event={"ID":"85326841-0a68-43aa-b88f-a1a37e02e64d","Type":"ContainerDied","Data":"1e676fe8599a563e7dca41efa81e3c17d4ba833663792b4203e92c80a29f756d"} Sep 09 07:02:01.908949 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.908918 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" Sep 09 07:02:01.908949 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.908932 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp" event={"ID":"85326841-0a68-43aa-b88f-a1a37e02e64d","Type":"ContainerDied","Data":"45676b97efcfad2421a245b9066c6bae3ca88a13950231f04b25639632c22419"} Sep 09 07:02:01.916195 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.916070 2559 scope.go:117] "RemoveContainer" containerID="c84a0a4a2eed825990e9dbc45e46cf9d1f5fb531bce3e1e84e9bc0055c2e5d9a" Sep 09 07:02:01.924069 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.924035 2559 scope.go:117] "RemoveContainer" containerID="37d82fb8f92b79b8985d0d1624d5b231a9b3fc4511c1786d8fa6c123ed6c9a60" Sep 09 07:02:01.929991 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.929968 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm"] Sep 09 07:02:01.932219 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.932203 2559 scope.go:117] "RemoveContainer" containerID="49d60e6006f45271cf791d4e1011ab7dc610c54aab27c3e7a1e7a42b08b0fbb9" Sep 09 07:02:01.935509 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.935486 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-6584dcf87c-j78lm"] Sep 09 07:02:01.939679 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.939663 2559 scope.go:117] "RemoveContainer" containerID="676b1113cc69b3f5e6395133a73bb54c9ed30b477054eb0302b8dd33eb7f8b28" Sep 09 07:02:01.939946 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:02:01.939929 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"676b1113cc69b3f5e6395133a73bb54c9ed30b477054eb0302b8dd33eb7f8b28\": container with ID starting with 676b1113cc69b3f5e6395133a73bb54c9ed30b477054eb0302b8dd33eb7f8b28 not found: ID does not exist" containerID="676b1113cc69b3f5e6395133a73bb54c9ed30b477054eb0302b8dd33eb7f8b28" Sep 09 07:02:01.940013 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.939958 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"676b1113cc69b3f5e6395133a73bb54c9ed30b477054eb0302b8dd33eb7f8b28"} err="failed to get container status \"676b1113cc69b3f5e6395133a73bb54c9ed30b477054eb0302b8dd33eb7f8b28\": rpc error: code = NotFound desc = could not find container \"676b1113cc69b3f5e6395133a73bb54c9ed30b477054eb0302b8dd33eb7f8b28\": container with ID starting with 676b1113cc69b3f5e6395133a73bb54c9ed30b477054eb0302b8dd33eb7f8b28 not found: ID does not exist" Sep 09 07:02:01.940013 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.939981 2559 scope.go:117] "RemoveContainer" containerID="c84a0a4a2eed825990e9dbc45e46cf9d1f5fb531bce3e1e84e9bc0055c2e5d9a" Sep 09 07:02:01.940260 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:02:01.940242 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c84a0a4a2eed825990e9dbc45e46cf9d1f5fb531bce3e1e84e9bc0055c2e5d9a\": container with ID starting with c84a0a4a2eed825990e9dbc45e46cf9d1f5fb531bce3e1e84e9bc0055c2e5d9a not found: ID does not exist" containerID="c84a0a4a2eed825990e9dbc45e46cf9d1f5fb531bce3e1e84e9bc0055c2e5d9a" Sep 09 07:02:01.940304 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.940265 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c84a0a4a2eed825990e9dbc45e46cf9d1f5fb531bce3e1e84e9bc0055c2e5d9a"} err="failed to get container status \"c84a0a4a2eed825990e9dbc45e46cf9d1f5fb531bce3e1e84e9bc0055c2e5d9a\": rpc error: code = NotFound desc = could not find container \"c84a0a4a2eed825990e9dbc45e46cf9d1f5fb531bce3e1e84e9bc0055c2e5d9a\": container with ID starting with c84a0a4a2eed825990e9dbc45e46cf9d1f5fb531bce3e1e84e9bc0055c2e5d9a not found: ID does not exist" Sep 09 07:02:01.940304 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.940279 2559 scope.go:117] "RemoveContainer" containerID="37d82fb8f92b79b8985d0d1624d5b231a9b3fc4511c1786d8fa6c123ed6c9a60" Sep 09 07:02:01.940503 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:02:01.940486 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37d82fb8f92b79b8985d0d1624d5b231a9b3fc4511c1786d8fa6c123ed6c9a60\": container with ID starting with 37d82fb8f92b79b8985d0d1624d5b231a9b3fc4511c1786d8fa6c123ed6c9a60 not found: ID does not exist" containerID="37d82fb8f92b79b8985d0d1624d5b231a9b3fc4511c1786d8fa6c123ed6c9a60" Sep 09 07:02:01.940549 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.940509 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37d82fb8f92b79b8985d0d1624d5b231a9b3fc4511c1786d8fa6c123ed6c9a60"} err="failed to get container status \"37d82fb8f92b79b8985d0d1624d5b231a9b3fc4511c1786d8fa6c123ed6c9a60\": rpc error: code = NotFound desc = could not find container \"37d82fb8f92b79b8985d0d1624d5b231a9b3fc4511c1786d8fa6c123ed6c9a60\": container with ID starting with 37d82fb8f92b79b8985d0d1624d5b231a9b3fc4511c1786d8fa6c123ed6c9a60 not found: ID does not exist" Sep 09 07:02:01.940549 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.940524 2559 scope.go:117] "RemoveContainer" containerID="49d60e6006f45271cf791d4e1011ab7dc610c54aab27c3e7a1e7a42b08b0fbb9" Sep 09 07:02:01.940744 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:02:01.940727 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49d60e6006f45271cf791d4e1011ab7dc610c54aab27c3e7a1e7a42b08b0fbb9\": container with ID starting with 49d60e6006f45271cf791d4e1011ab7dc610c54aab27c3e7a1e7a42b08b0fbb9 not found: ID does not exist" containerID="49d60e6006f45271cf791d4e1011ab7dc610c54aab27c3e7a1e7a42b08b0fbb9" Sep 09 07:02:01.940815 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.940751 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49d60e6006f45271cf791d4e1011ab7dc610c54aab27c3e7a1e7a42b08b0fbb9"} err="failed to get container status \"49d60e6006f45271cf791d4e1011ab7dc610c54aab27c3e7a1e7a42b08b0fbb9\": rpc error: code = NotFound desc = could not find container \"49d60e6006f45271cf791d4e1011ab7dc610c54aab27c3e7a1e7a42b08b0fbb9\": container with ID starting with 49d60e6006f45271cf791d4e1011ab7dc610c54aab27c3e7a1e7a42b08b0fbb9 not found: ID does not exist" Sep 09 07:02:01.940815 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.940772 2559 scope.go:117] "RemoveContainer" containerID="595340ef7c96890b83629f8b5b02d98c21232abc316ee4388f5054982f90aaee" Sep 09 07:02:01.946624 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.946602 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp"] Sep 09 07:02:01.948552 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.948536 2559 scope.go:117] "RemoveContainer" containerID="1e676fe8599a563e7dca41efa81e3c17d4ba833663792b4203e92c80a29f756d" Sep 09 07:02:01.952876 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.952854 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7b99577c94-jwlvp"] Sep 09 07:02:01.956907 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.956893 2559 scope.go:117] "RemoveContainer" containerID="a6634df0650f0679bb5a23532bd5a714c4c71365288aad1c5336d3b4ca7f930d" Sep 09 07:02:01.964098 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.964083 2559 scope.go:117] "RemoveContainer" containerID="16ec7b10d7ba9084ed703eba4b9a5cb36aa2f72a963fbc4168a091672ae0dac7" Sep 09 07:02:01.970921 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.970904 2559 scope.go:117] "RemoveContainer" containerID="595340ef7c96890b83629f8b5b02d98c21232abc316ee4388f5054982f90aaee" Sep 09 07:02:01.971193 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:02:01.971173 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"595340ef7c96890b83629f8b5b02d98c21232abc316ee4388f5054982f90aaee\": container with ID starting with 595340ef7c96890b83629f8b5b02d98c21232abc316ee4388f5054982f90aaee not found: ID does not exist" containerID="595340ef7c96890b83629f8b5b02d98c21232abc316ee4388f5054982f90aaee" Sep 09 07:02:01.971272 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.971201 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"595340ef7c96890b83629f8b5b02d98c21232abc316ee4388f5054982f90aaee"} err="failed to get container status \"595340ef7c96890b83629f8b5b02d98c21232abc316ee4388f5054982f90aaee\": rpc error: code = NotFound desc = could not find container \"595340ef7c96890b83629f8b5b02d98c21232abc316ee4388f5054982f90aaee\": container with ID starting with 595340ef7c96890b83629f8b5b02d98c21232abc316ee4388f5054982f90aaee not found: ID does not exist" Sep 09 07:02:01.971272 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.971223 2559 scope.go:117] "RemoveContainer" containerID="1e676fe8599a563e7dca41efa81e3c17d4ba833663792b4203e92c80a29f756d" Sep 09 07:02:01.971474 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:02:01.971456 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e676fe8599a563e7dca41efa81e3c17d4ba833663792b4203e92c80a29f756d\": container with ID starting with 1e676fe8599a563e7dca41efa81e3c17d4ba833663792b4203e92c80a29f756d not found: ID does not exist" containerID="1e676fe8599a563e7dca41efa81e3c17d4ba833663792b4203e92c80a29f756d" Sep 09 07:02:01.971524 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.971481 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e676fe8599a563e7dca41efa81e3c17d4ba833663792b4203e92c80a29f756d"} err="failed to get container status \"1e676fe8599a563e7dca41efa81e3c17d4ba833663792b4203e92c80a29f756d\": rpc error: code = NotFound desc = could not find container \"1e676fe8599a563e7dca41efa81e3c17d4ba833663792b4203e92c80a29f756d\": container with ID starting with 1e676fe8599a563e7dca41efa81e3c17d4ba833663792b4203e92c80a29f756d not found: ID does not exist" Sep 09 07:02:01.971524 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.971497 2559 scope.go:117] "RemoveContainer" containerID="a6634df0650f0679bb5a23532bd5a714c4c71365288aad1c5336d3b4ca7f930d" Sep 09 07:02:01.971695 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:02:01.971682 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6634df0650f0679bb5a23532bd5a714c4c71365288aad1c5336d3b4ca7f930d\": container with ID starting with a6634df0650f0679bb5a23532bd5a714c4c71365288aad1c5336d3b4ca7f930d not found: ID does not exist" containerID="a6634df0650f0679bb5a23532bd5a714c4c71365288aad1c5336d3b4ca7f930d" Sep 09 07:02:01.971737 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.971698 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6634df0650f0679bb5a23532bd5a714c4c71365288aad1c5336d3b4ca7f930d"} err="failed to get container status \"a6634df0650f0679bb5a23532bd5a714c4c71365288aad1c5336d3b4ca7f930d\": rpc error: code = NotFound desc = could not find container \"a6634df0650f0679bb5a23532bd5a714c4c71365288aad1c5336d3b4ca7f930d\": container with ID starting with a6634df0650f0679bb5a23532bd5a714c4c71365288aad1c5336d3b4ca7f930d not found: ID does not exist" Sep 09 07:02:01.971737 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.971710 2559 scope.go:117] "RemoveContainer" containerID="16ec7b10d7ba9084ed703eba4b9a5cb36aa2f72a963fbc4168a091672ae0dac7" Sep 09 07:02:01.971924 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:02:01.971903 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16ec7b10d7ba9084ed703eba4b9a5cb36aa2f72a963fbc4168a091672ae0dac7\": container with ID starting with 16ec7b10d7ba9084ed703eba4b9a5cb36aa2f72a963fbc4168a091672ae0dac7 not found: ID does not exist" containerID="16ec7b10d7ba9084ed703eba4b9a5cb36aa2f72a963fbc4168a091672ae0dac7" Sep 09 07:02:01.972015 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:01.971925 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16ec7b10d7ba9084ed703eba4b9a5cb36aa2f72a963fbc4168a091672ae0dac7"} err="failed to get container status \"16ec7b10d7ba9084ed703eba4b9a5cb36aa2f72a963fbc4168a091672ae0dac7\": rpc error: code = NotFound desc = could not find container \"16ec7b10d7ba9084ed703eba4b9a5cb36aa2f72a963fbc4168a091672ae0dac7\": container with ID starting with 16ec7b10d7ba9084ed703eba4b9a5cb36aa2f72a963fbc4168a091672ae0dac7 not found: ID does not exist" Sep 09 07:02:02.701725 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:02.701686 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ea79a6d-5d15-49c8-8bfc-aa68923db82d" path="/var/lib/kubelet/pods/0ea79a6d-5d15-49c8-8bfc-aa68923db82d/volumes" Sep 09 07:02:02.702223 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:02.702206 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85326841-0a68-43aa-b88f-a1a37e02e64d" path="/var/lib/kubelet/pods/85326841-0a68-43aa-b88f-a1a37e02e64d/volumes" Sep 09 07:02:09.450981 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.450946 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld"] Sep 09 07:02:09.451436 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.451331 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfc3cd85-3e6a-4ee2-bee8-c13375ef60eb" containerName="postgres" Sep 09 07:02:09.451436 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.451344 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfc3cd85-3e6a-4ee2-bee8-c13375ef60eb" containerName="postgres" Sep 09 07:02:09.451436 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.451352 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ea79a6d-5d15-49c8-8bfc-aa68923db82d" containerName="feast-init" Sep 09 07:02:09.451436 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.451358 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ea79a6d-5d15-49c8-8bfc-aa68923db82d" containerName="feast-init" Sep 09 07:02:09.451436 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.451365 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85326841-0a68-43aa-b88f-a1a37e02e64d" containerName="online" Sep 09 07:02:09.451436 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.451373 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="85326841-0a68-43aa-b88f-a1a37e02e64d" containerName="online" Sep 09 07:02:09.451436 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.451385 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ea79a6d-5d15-49c8-8bfc-aa68923db82d" containerName="online" Sep 09 07:02:09.451436 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.451392 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ea79a6d-5d15-49c8-8bfc-aa68923db82d" containerName="online" Sep 09 07:02:09.451436 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.451400 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ea79a6d-5d15-49c8-8bfc-aa68923db82d" containerName="registry" Sep 09 07:02:09.451436 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.451405 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ea79a6d-5d15-49c8-8bfc-aa68923db82d" containerName="registry" Sep 09 07:02:09.451436 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.451413 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04f835e9-9bba-4530-8ca2-4094714cf65b" containerName="redis" Sep 09 07:02:09.451436 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.451419 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="04f835e9-9bba-4530-8ca2-4094714cf65b" containerName="redis" Sep 09 07:02:09.451436 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.451427 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85326841-0a68-43aa-b88f-a1a37e02e64d" containerName="feast-apply" Sep 09 07:02:09.451436 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.451432 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="85326841-0a68-43aa-b88f-a1a37e02e64d" containerName="feast-apply" Sep 09 07:02:09.451436 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.451439 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85326841-0a68-43aa-b88f-a1a37e02e64d" containerName="registry" Sep 09 07:02:09.451436 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.451444 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="85326841-0a68-43aa-b88f-a1a37e02e64d" containerName="registry" Sep 09 07:02:09.451921 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.451456 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="247e9f69-e45c-49e7-be80-42c2ad4e0e0f" containerName="manager" Sep 09 07:02:09.451921 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.451464 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="247e9f69-e45c-49e7-be80-42c2ad4e0e0f" containerName="manager" Sep 09 07:02:09.451921 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.451469 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ea79a6d-5d15-49c8-8bfc-aa68923db82d" containerName="feast-apply" Sep 09 07:02:09.451921 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.451474 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ea79a6d-5d15-49c8-8bfc-aa68923db82d" containerName="feast-apply" Sep 09 07:02:09.451921 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.451489 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85326841-0a68-43aa-b88f-a1a37e02e64d" containerName="feast-init" Sep 09 07:02:09.451921 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.451494 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="85326841-0a68-43aa-b88f-a1a37e02e64d" containerName="feast-init" Sep 09 07:02:09.451921 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.451548 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ea79a6d-5d15-49c8-8bfc-aa68923db82d" containerName="online" Sep 09 07:02:09.451921 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.451558 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="247e9f69-e45c-49e7-be80-42c2ad4e0e0f" containerName="manager" Sep 09 07:02:09.451921 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.451564 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfc3cd85-3e6a-4ee2-bee8-c13375ef60eb" containerName="postgres" Sep 09 07:02:09.451921 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.451569 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ea79a6d-5d15-49c8-8bfc-aa68923db82d" containerName="registry" Sep 09 07:02:09.451921 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.451575 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="04f835e9-9bba-4530-8ca2-4094714cf65b" containerName="redis" Sep 09 07:02:09.451921 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.451581 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="85326841-0a68-43aa-b88f-a1a37e02e64d" containerName="registry" Sep 09 07:02:09.451921 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.451588 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="85326841-0a68-43aa-b88f-a1a37e02e64d" containerName="online" Sep 09 07:02:09.457039 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.457020 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:02:09.459391 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.459369 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 09 07:02:09.459391 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.459381 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 09 07:02:09.459556 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.459388 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-55d8t\"" Sep 09 07:02:09.459556 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.459371 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 09 07:02:09.459745 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.459729 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 09 07:02:09.459866 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.459851 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 09 07:02:09.460204 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.460190 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 09 07:02:09.467368 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.467346 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld"] Sep 09 07:02:09.540246 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.540220 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/54aa0daf-e6f1-408d-b78d-3f4110d28df5-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-4vjld\" (UID: \"54aa0daf-e6f1-408d-b78d-3f4110d28df5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:02:09.540386 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.540264 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/54aa0daf-e6f1-408d-b78d-3f4110d28df5-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-4vjld\" (UID: \"54aa0daf-e6f1-408d-b78d-3f4110d28df5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:02:09.540386 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.540281 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/54aa0daf-e6f1-408d-b78d-3f4110d28df5-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-4vjld\" (UID: \"54aa0daf-e6f1-408d-b78d-3f4110d28df5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:02:09.540386 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.540356 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/54aa0daf-e6f1-408d-b78d-3f4110d28df5-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-4vjld\" (UID: \"54aa0daf-e6f1-408d-b78d-3f4110d28df5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:02:09.540548 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.540432 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/54aa0daf-e6f1-408d-b78d-3f4110d28df5-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-4vjld\" (UID: \"54aa0daf-e6f1-408d-b78d-3f4110d28df5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:02:09.540548 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.540478 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6s7z\" (UniqueName: \"kubernetes.io/projected/54aa0daf-e6f1-408d-b78d-3f4110d28df5-kube-api-access-b6s7z\") pod \"feast-simple-feast-setup-57576b8fcf-4vjld\" (UID: \"54aa0daf-e6f1-408d-b78d-3f4110d28df5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:02:09.640967 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.640937 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6s7z\" (UniqueName: \"kubernetes.io/projected/54aa0daf-e6f1-408d-b78d-3f4110d28df5-kube-api-access-b6s7z\") pod \"feast-simple-feast-setup-57576b8fcf-4vjld\" (UID: \"54aa0daf-e6f1-408d-b78d-3f4110d28df5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:02:09.641155 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.640990 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/54aa0daf-e6f1-408d-b78d-3f4110d28df5-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-4vjld\" (UID: \"54aa0daf-e6f1-408d-b78d-3f4110d28df5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:02:09.641155 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.641033 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/54aa0daf-e6f1-408d-b78d-3f4110d28df5-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-4vjld\" (UID: \"54aa0daf-e6f1-408d-b78d-3f4110d28df5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:02:09.641155 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.641075 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/54aa0daf-e6f1-408d-b78d-3f4110d28df5-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-4vjld\" (UID: \"54aa0daf-e6f1-408d-b78d-3f4110d28df5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:02:09.641337 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:02:09.641159 2559 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Sep 09 07:02:09.641337 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.641202 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/54aa0daf-e6f1-408d-b78d-3f4110d28df5-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-4vjld\" (UID: \"54aa0daf-e6f1-408d-b78d-3f4110d28df5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:02:09.641337 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:02:09.641218 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54aa0daf-e6f1-408d-b78d-3f4110d28df5-offline-tls podName:54aa0daf-e6f1-408d-b78d-3f4110d28df5 nodeName:}" failed. No retries permitted until 2026-09-09 07:02:10.141201808 +0000 UTC m=+1428.037619208 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/54aa0daf-e6f1-408d-b78d-3f4110d28df5-offline-tls") pod "feast-simple-feast-setup-57576b8fcf-4vjld" (UID: "54aa0daf-e6f1-408d-b78d-3f4110d28df5") : secret "feast-simple-feast-setup-offline-tls" not found Sep 09 07:02:09.641337 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.641284 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/54aa0daf-e6f1-408d-b78d-3f4110d28df5-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-4vjld\" (UID: \"54aa0daf-e6f1-408d-b78d-3f4110d28df5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:02:09.641609 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:02:09.641361 2559 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Sep 09 07:02:09.641609 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:02:09.641398 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54aa0daf-e6f1-408d-b78d-3f4110d28df5-ui-tls podName:54aa0daf-e6f1-408d-b78d-3f4110d28df5 nodeName:}" failed. No retries permitted until 2026-09-09 07:02:10.141388117 +0000 UTC m=+1428.037805516 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/54aa0daf-e6f1-408d-b78d-3f4110d28df5-ui-tls") pod "feast-simple-feast-setup-57576b8fcf-4vjld" (UID: "54aa0daf-e6f1-408d-b78d-3f4110d28df5") : secret "feast-simple-feast-setup-ui-tls" not found Sep 09 07:02:09.641609 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.641443 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/54aa0daf-e6f1-408d-b78d-3f4110d28df5-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-4vjld\" (UID: \"54aa0daf-e6f1-408d-b78d-3f4110d28df5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:02:09.643511 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.643487 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/54aa0daf-e6f1-408d-b78d-3f4110d28df5-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-4vjld\" (UID: \"54aa0daf-e6f1-408d-b78d-3f4110d28df5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:02:09.643607 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.643524 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/54aa0daf-e6f1-408d-b78d-3f4110d28df5-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-4vjld\" (UID: \"54aa0daf-e6f1-408d-b78d-3f4110d28df5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:02:09.649910 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:09.649879 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6s7z\" (UniqueName: \"kubernetes.io/projected/54aa0daf-e6f1-408d-b78d-3f4110d28df5-kube-api-access-b6s7z\") pod \"feast-simple-feast-setup-57576b8fcf-4vjld\" (UID: \"54aa0daf-e6f1-408d-b78d-3f4110d28df5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:02:10.144924 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:10.144885 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/54aa0daf-e6f1-408d-b78d-3f4110d28df5-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-4vjld\" (UID: \"54aa0daf-e6f1-408d-b78d-3f4110d28df5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:02:10.145131 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:10.144979 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/54aa0daf-e6f1-408d-b78d-3f4110d28df5-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-4vjld\" (UID: \"54aa0daf-e6f1-408d-b78d-3f4110d28df5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:02:10.147406 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:10.147374 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/54aa0daf-e6f1-408d-b78d-3f4110d28df5-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-4vjld\" (UID: \"54aa0daf-e6f1-408d-b78d-3f4110d28df5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:02:10.147527 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:10.147430 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/54aa0daf-e6f1-408d-b78d-3f4110d28df5-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-4vjld\" (UID: \"54aa0daf-e6f1-408d-b78d-3f4110d28df5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:02:10.376198 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:10.376166 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:02:10.656468 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:10.656334 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld"] Sep 09 07:02:10.658321 ip-10-0-132-137 kubenswrapper[2559]: W0909 07:02:10.658284 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod54aa0daf_e6f1_408d_b78d_3f4110d28df5.slice/crio-2adf4304f28fefaf5a59581420f9835fb21aac9892611fdd8f8627e3cbf6ca39 WatchSource:0}: Error finding container 2adf4304f28fefaf5a59581420f9835fb21aac9892611fdd8f8627e3cbf6ca39: Status 404 returned error can't find the container with id 2adf4304f28fefaf5a59581420f9835fb21aac9892611fdd8f8627e3cbf6ca39 Sep 09 07:02:10.660647 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:10.660626 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 07:02:10.956972 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:10.956935 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" event={"ID":"54aa0daf-e6f1-408d-b78d-3f4110d28df5","Type":"ContainerStarted","Data":"2adf4304f28fefaf5a59581420f9835fb21aac9892611fdd8f8627e3cbf6ca39"} Sep 09 07:02:26.049940 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:26.049899 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" event={"ID":"54aa0daf-e6f1-408d-b78d-3f4110d28df5","Type":"ContainerStarted","Data":"3328f0764fe6f1bb38c2d30f0659f271fc7cea715d7ceba208f87b66d880a9f0"} Sep 09 07:02:31.074207 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:31.074167 2559 generic.go:356] "Generic (PLEG): container finished" podID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerID="3328f0764fe6f1bb38c2d30f0659f271fc7cea715d7ceba208f87b66d880a9f0" exitCode=0 Sep 09 07:02:31.074609 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:31.074217 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" event={"ID":"54aa0daf-e6f1-408d-b78d-3f4110d28df5","Type":"ContainerDied","Data":"3328f0764fe6f1bb38c2d30f0659f271fc7cea715d7ceba208f87b66d880a9f0"} Sep 09 07:02:32.079910 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:32.079875 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" event={"ID":"54aa0daf-e6f1-408d-b78d-3f4110d28df5","Type":"ContainerStarted","Data":"d04fd9a71e218cbe6f0ffa469961062aa7775210a3fcf26fe2ccd9d8c87cce94"} Sep 09 07:02:37.104762 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:37.104680 2559 generic.go:356] "Generic (PLEG): container finished" podID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerID="d04fd9a71e218cbe6f0ffa469961062aa7775210a3fcf26fe2ccd9d8c87cce94" exitCode=0 Sep 09 07:02:37.104762 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:37.104751 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" event={"ID":"54aa0daf-e6f1-408d-b78d-3f4110d28df5","Type":"ContainerDied","Data":"d04fd9a71e218cbe6f0ffa469961062aa7775210a3fcf26fe2ccd9d8c87cce94"} Sep 09 07:02:38.116127 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:38.116080 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" event={"ID":"54aa0daf-e6f1-408d-b78d-3f4110d28df5","Type":"ContainerStarted","Data":"31c2b88519eae736d24de98ce67228b77729f4b6fd9c21308106edd505d7174f"} Sep 09 07:02:38.116616 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:38.116152 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" event={"ID":"54aa0daf-e6f1-408d-b78d-3f4110d28df5","Type":"ContainerStarted","Data":"dcfacab6ccf95bf050957b23a4258bc51a6f85a2fca721daef8177428b6defda"} Sep 09 07:02:38.116616 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:38.116176 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" event={"ID":"54aa0daf-e6f1-408d-b78d-3f4110d28df5","Type":"ContainerStarted","Data":"9b132fec88521093e07a4c77954a4b935c065aa0fa3d1adc0f69a9c573ac4182"} Sep 09 07:02:38.116616 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:38.116194 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" event={"ID":"54aa0daf-e6f1-408d-b78d-3f4110d28df5","Type":"ContainerStarted","Data":"66f65bfb0a9bb827b80f220d1fdc2be858effdb934dcce279daf7641999fe085"} Sep 09 07:02:38.141920 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:38.141848 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" podStartSLOduration=14.250878826 podStartE2EDuration="29.141811143s" totalImagesPullingTime="14.890932317s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:02:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:02:38.140989288 +0000 UTC m=+1456.037406712" watchObservedRunningTime="2026-09-09 07:02:38.141811143 +0000 UTC m=+1456.038228566" Sep 09 07:02:40.377176 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:40.377132 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:02:40.377693 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:40.377382 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:02:40.377693 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:40.377427 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:02:40.377693 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:40.377447 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:02:40.377693 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:40.377466 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:02:40.377693 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:40.377484 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:02:40.377960 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:40.377810 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:02:40.377960 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:40.377854 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:02:40.379703 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:40.379658 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" podUID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.40:8816: connect: connection refused" Sep 09 07:02:40.379826 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:40.379725 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" podUID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.40:8443: connect: connection refused" Sep 09 07:02:40.379826 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:40.379750 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" podUID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerName="online" probeResult="failure" output="Get \"https://10.132.0.40:6567/health\": dial tcp 10.132.0.40:6567: connect: connection refused" Sep 09 07:02:40.380101 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:40.380067 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" podUID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.40:6571: connect: connection refused" Sep 09 07:02:43.377464 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:43.377417 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" podUID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.40:8816: connect: connection refused" Sep 09 07:02:43.378202 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:43.377417 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" podUID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.40:8443: connect: connection refused" Sep 09 07:02:43.378400 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:43.377542 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" podUID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerName="online" probeResult="failure" output="Get \"https://10.132.0.40:6567/health\": dial tcp 10.132.0.40:6567: connect: connection refused" Sep 09 07:02:43.378400 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:43.377460 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" podUID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.40:6571: connect: connection refused" Sep 09 07:02:46.377187 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:46.377140 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:02:46.377584 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:46.377445 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:02:46.377584 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:46.377583 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:02:46.378255 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:46.378217 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:02:46.378571 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:46.378550 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:02:46.379176 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:46.379158 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:02:46.382890 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:46.382874 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:02:47.171170 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:02:47.171131 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:03:22.586936 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:22.586905 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nf9gf_26d28788-4f98-454b-ab39-2d0d857aa674/console-operator/2.log" Sep 09 07:03:22.591549 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:22.591521 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nf9gf_26d28788-4f98-454b-ab39-2d0d857aa674/console-operator/2.log" Sep 09 07:03:48.884776 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:48.884688 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld"] Sep 09 07:03:48.885314 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:48.884937 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" podUID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerName="registry" containerID="cri-o://66f65bfb0a9bb827b80f220d1fdc2be858effdb934dcce279daf7641999fe085" gracePeriod=30 Sep 09 07:03:48.885314 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:48.884975 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" podUID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerName="offline" containerID="cri-o://dcfacab6ccf95bf050957b23a4258bc51a6f85a2fca721daef8177428b6defda" gracePeriod=30 Sep 09 07:03:48.885314 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:48.885003 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" podUID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerName="ui" containerID="cri-o://31c2b88519eae736d24de98ce67228b77729f4b6fd9c21308106edd505d7174f" gracePeriod=30 Sep 09 07:03:48.885314 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:48.885004 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" podUID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerName="online" containerID="cri-o://9b132fec88521093e07a4c77954a4b935c065aa0fa3d1adc0f69a9c573ac4182" gracePeriod=30 Sep 09 07:03:49.362781 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:49.362742 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx"] Sep 09 07:03:49.366549 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:49.366529 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:03:49.368747 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:49.368722 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-pkghr\"" Sep 09 07:03:49.378687 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:49.378662 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx"] Sep 09 07:03:49.492254 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:49.492216 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b4f53b83-9137-4998-b3b2-bea46e694595-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-q5fpx\" (UID: \"b4f53b83-9137-4998-b3b2-bea46e694595\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:03:49.492254 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:49.492263 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6rdz\" (UniqueName: \"kubernetes.io/projected/b4f53b83-9137-4998-b3b2-bea46e694595-kube-api-access-d6rdz\") pod \"feast-simple-feast-setup-57576b8fcf-q5fpx\" (UID: \"b4f53b83-9137-4998-b3b2-bea46e694595\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:03:49.492480 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:49.492362 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b4f53b83-9137-4998-b3b2-bea46e694595-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-q5fpx\" (UID: \"b4f53b83-9137-4998-b3b2-bea46e694595\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:03:49.492480 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:49.492422 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b4f53b83-9137-4998-b3b2-bea46e694595-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-q5fpx\" (UID: \"b4f53b83-9137-4998-b3b2-bea46e694595\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:03:49.492566 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:49.492490 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b4f53b83-9137-4998-b3b2-bea46e694595-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-q5fpx\" (UID: \"b4f53b83-9137-4998-b3b2-bea46e694595\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:03:49.492566 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:49.492550 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b4f53b83-9137-4998-b3b2-bea46e694595-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-q5fpx\" (UID: \"b4f53b83-9137-4998-b3b2-bea46e694595\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:03:49.593834 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:49.593800 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b4f53b83-9137-4998-b3b2-bea46e694595-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-q5fpx\" (UID: \"b4f53b83-9137-4998-b3b2-bea46e694595\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:03:49.594018 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:49.593854 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b4f53b83-9137-4998-b3b2-bea46e694595-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-q5fpx\" (UID: \"b4f53b83-9137-4998-b3b2-bea46e694595\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:03:49.594018 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:03:49.593946 2559 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found Sep 09 07:03:49.594018 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:03:49.593996 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4f53b83-9137-4998-b3b2-bea46e694595-online-tls podName:b4f53b83-9137-4998-b3b2-bea46e694595 nodeName:}" failed. No retries permitted until 2026-09-09 07:03:50.093980459 +0000 UTC m=+1527.990397859 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/b4f53b83-9137-4998-b3b2-bea46e694595-online-tls") pod "feast-simple-feast-setup-57576b8fcf-q5fpx" (UID: "b4f53b83-9137-4998-b3b2-bea46e694595") : secret "feast-simple-feast-setup-online-tls" not found Sep 09 07:03:49.594018 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:49.593986 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b4f53b83-9137-4998-b3b2-bea46e694595-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-q5fpx\" (UID: \"b4f53b83-9137-4998-b3b2-bea46e694595\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:03:49.594274 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:49.594030 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6rdz\" (UniqueName: \"kubernetes.io/projected/b4f53b83-9137-4998-b3b2-bea46e694595-kube-api-access-d6rdz\") pod \"feast-simple-feast-setup-57576b8fcf-q5fpx\" (UID: \"b4f53b83-9137-4998-b3b2-bea46e694595\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:03:49.594274 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:49.594078 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b4f53b83-9137-4998-b3b2-bea46e694595-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-q5fpx\" (UID: \"b4f53b83-9137-4998-b3b2-bea46e694595\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:03:49.594274 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:49.594119 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b4f53b83-9137-4998-b3b2-bea46e694595-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-q5fpx\" (UID: \"b4f53b83-9137-4998-b3b2-bea46e694595\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:03:49.594943 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:03:49.594748 2559 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Sep 09 07:03:49.595099 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:49.594989 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b4f53b83-9137-4998-b3b2-bea46e694595-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-q5fpx\" (UID: \"b4f53b83-9137-4998-b3b2-bea46e694595\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:03:49.595173 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:03:49.595096 2559 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Sep 09 07:03:49.599077 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:03:49.595490 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4f53b83-9137-4998-b3b2-bea46e694595-offline-tls podName:b4f53b83-9137-4998-b3b2-bea46e694595 nodeName:}" failed. No retries permitted until 2026-09-09 07:03:50.095456956 +0000 UTC m=+1527.991874378 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/b4f53b83-9137-4998-b3b2-bea46e694595-offline-tls") pod "feast-simple-feast-setup-57576b8fcf-q5fpx" (UID: "b4f53b83-9137-4998-b3b2-bea46e694595") : secret "feast-simple-feast-setup-offline-tls" not found Sep 09 07:03:49.599077 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:03:49.595758 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4f53b83-9137-4998-b3b2-bea46e694595-registry-tls podName:b4f53b83-9137-4998-b3b2-bea46e694595 nodeName:}" failed. No retries permitted until 2026-09-09 07:03:50.095730089 +0000 UTC m=+1527.992147505 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/b4f53b83-9137-4998-b3b2-bea46e694595-registry-tls") pod "feast-simple-feast-setup-57576b8fcf-q5fpx" (UID: "b4f53b83-9137-4998-b3b2-bea46e694595") : secret "feast-simple-feast-setup-registry-tls" not found Sep 09 07:03:49.603381 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:49.603349 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b4f53b83-9137-4998-b3b2-bea46e694595-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-q5fpx\" (UID: \"b4f53b83-9137-4998-b3b2-bea46e694595\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:03:49.603561 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:49.603542 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6rdz\" (UniqueName: \"kubernetes.io/projected/b4f53b83-9137-4998-b3b2-bea46e694595-kube-api-access-d6rdz\") pod \"feast-simple-feast-setup-57576b8fcf-q5fpx\" (UID: \"b4f53b83-9137-4998-b3b2-bea46e694595\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:03:50.098359 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:50.098272 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b4f53b83-9137-4998-b3b2-bea46e694595-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-q5fpx\" (UID: \"b4f53b83-9137-4998-b3b2-bea46e694595\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:03:50.098359 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:50.098331 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b4f53b83-9137-4998-b3b2-bea46e694595-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-q5fpx\" (UID: \"b4f53b83-9137-4998-b3b2-bea46e694595\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:03:50.098856 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:50.098367 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b4f53b83-9137-4998-b3b2-bea46e694595-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-q5fpx\" (UID: \"b4f53b83-9137-4998-b3b2-bea46e694595\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:03:50.100872 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:50.100840 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b4f53b83-9137-4998-b3b2-bea46e694595-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-q5fpx\" (UID: \"b4f53b83-9137-4998-b3b2-bea46e694595\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:03:50.100872 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:50.100857 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b4f53b83-9137-4998-b3b2-bea46e694595-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-q5fpx\" (UID: \"b4f53b83-9137-4998-b3b2-bea46e694595\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:03:50.101015 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:50.100884 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b4f53b83-9137-4998-b3b2-bea46e694595-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-q5fpx\" (UID: \"b4f53b83-9137-4998-b3b2-bea46e694595\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:03:50.276071 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:50.276013 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:03:50.432669 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:50.432634 2559 generic.go:356] "Generic (PLEG): container finished" podID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerID="31c2b88519eae736d24de98ce67228b77729f4b6fd9c21308106edd505d7174f" exitCode=0 Sep 09 07:03:50.432669 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:50.432660 2559 generic.go:356] "Generic (PLEG): container finished" podID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerID="9b132fec88521093e07a4c77954a4b935c065aa0fa3d1adc0f69a9c573ac4182" exitCode=0 Sep 09 07:03:50.432875 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:50.432707 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" event={"ID":"54aa0daf-e6f1-408d-b78d-3f4110d28df5","Type":"ContainerDied","Data":"31c2b88519eae736d24de98ce67228b77729f4b6fd9c21308106edd505d7174f"} Sep 09 07:03:50.432875 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:50.432749 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" event={"ID":"54aa0daf-e6f1-408d-b78d-3f4110d28df5","Type":"ContainerDied","Data":"9b132fec88521093e07a4c77954a4b935c065aa0fa3d1adc0f69a9c573ac4182"} Sep 09 07:03:50.553385 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:50.553357 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx"] Sep 09 07:03:50.555045 ip-10-0-132-137 kubenswrapper[2559]: W0909 07:03:50.555019 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb4f53b83_9137_4998_b3b2_bea46e694595.slice/crio-553f687b187f6ff0f93d5db51851c4ae974af33d9ede0c62fe2d815a323054b9 WatchSource:0}: Error finding container 553f687b187f6ff0f93d5db51851c4ae974af33d9ede0c62fe2d815a323054b9: Status 404 returned error can't find the container with id 553f687b187f6ff0f93d5db51851c4ae974af33d9ede0c62fe2d815a323054b9 Sep 09 07:03:51.439190 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:51.439147 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" event={"ID":"b4f53b83-9137-4998-b3b2-bea46e694595","Type":"ContainerStarted","Data":"6b9812a8a055e2ea219e16b6989eb6ebe9dcc64c4e3ce9262e52f568757444bc"} Sep 09 07:03:51.439586 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:51.439200 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" event={"ID":"b4f53b83-9137-4998-b3b2-bea46e694595","Type":"ContainerStarted","Data":"553f687b187f6ff0f93d5db51851c4ae974af33d9ede0c62fe2d815a323054b9"} Sep 09 07:03:55.460377 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:55.460347 2559 generic.go:356] "Generic (PLEG): container finished" podID="b4f53b83-9137-4998-b3b2-bea46e694595" containerID="6b9812a8a055e2ea219e16b6989eb6ebe9dcc64c4e3ce9262e52f568757444bc" exitCode=0 Sep 09 07:03:55.460689 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:55.460387 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" event={"ID":"b4f53b83-9137-4998-b3b2-bea46e694595","Type":"ContainerDied","Data":"6b9812a8a055e2ea219e16b6989eb6ebe9dcc64c4e3ce9262e52f568757444bc"} Sep 09 07:03:56.378332 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:56.378281 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" podUID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.40:8443: connect: connection refused" Sep 09 07:03:56.467553 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:56.467520 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" event={"ID":"b4f53b83-9137-4998-b3b2-bea46e694595","Type":"ContainerStarted","Data":"407ad025e050c34b5c57c5ca183939646cd6412a9d7f135d8a92afd9b9075018"} Sep 09 07:03:57.168423 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:03:57.168380 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" podUID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerName="online" probeResult="failure" output="Get \"https://10.132.0.40:6567/health\": dial tcp 10.132.0.40:6567: connect: connection refused" Sep 09 07:04:01.491900 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:01.491863 2559 generic.go:356] "Generic (PLEG): container finished" podID="b4f53b83-9137-4998-b3b2-bea46e694595" containerID="407ad025e050c34b5c57c5ca183939646cd6412a9d7f135d8a92afd9b9075018" exitCode=0 Sep 09 07:04:01.492320 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:01.491908 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" event={"ID":"b4f53b83-9137-4998-b3b2-bea46e694595","Type":"ContainerDied","Data":"407ad025e050c34b5c57c5ca183939646cd6412a9d7f135d8a92afd9b9075018"} Sep 09 07:04:02.503172 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:02.503128 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" event={"ID":"b4f53b83-9137-4998-b3b2-bea46e694595","Type":"ContainerStarted","Data":"16df504799e061bdbce19d9f30879c994b26ee7198bac76dd0d88c4e057d9897"} Sep 09 07:04:02.503689 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:02.503195 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" event={"ID":"b4f53b83-9137-4998-b3b2-bea46e694595","Type":"ContainerStarted","Data":"98d4b97907afa776bdd9d6ae7ea642008a953f94f1b4631192d1fcb783fb30f2"} Sep 09 07:04:02.503689 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:02.503216 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" event={"ID":"b4f53b83-9137-4998-b3b2-bea46e694595","Type":"ContainerStarted","Data":"6bc5cd7d29647903ebbb1d24b4c50aaa93b3c8d6c224b93b75d5310d256fcd0c"} Sep 09 07:04:02.503689 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:02.503234 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" event={"ID":"b4f53b83-9137-4998-b3b2-bea46e694595","Type":"ContainerStarted","Data":"e2568c0847b7d49e0b4d754bce7d196caff0ff03feaa4684878a1c67e8659469"} Sep 09 07:04:02.531509 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:02.531434 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" podStartSLOduration=13.531412809999999 podStartE2EDuration="13.53141281s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:03:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:04:02.530416027 +0000 UTC m=+1540.426833449" watchObservedRunningTime="2026-09-09 07:04:02.53141281 +0000 UTC m=+1540.427830232" Sep 09 07:04:05.276652 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:05.276614 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:04:05.277184 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:05.276678 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:04:05.277184 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:05.276697 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:04:05.277184 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:05.276715 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:04:05.278859 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:05.278819 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" podUID="b4f53b83-9137-4998-b3b2-bea46e694595" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.41:8443: connect: connection refused" Sep 09 07:04:05.279095 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:05.278858 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" podUID="b4f53b83-9137-4998-b3b2-bea46e694595" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.41:8816: connect: connection refused" Sep 09 07:04:05.279214 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:05.278865 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" podUID="b4f53b83-9137-4998-b3b2-bea46e694595" containerName="online" probeResult="failure" output="Get \"https://10.132.0.41:6567/health\": dial tcp 10.132.0.41:6567: connect: connection refused" Sep 09 07:04:05.279214 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:05.278879 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" podUID="b4f53b83-9137-4998-b3b2-bea46e694595" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.41:6571: connect: connection refused" Sep 09 07:04:06.378866 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:06.378739 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" podUID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.40:8443: connect: connection refused" Sep 09 07:04:07.169144 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:07.169090 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" podUID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerName="online" probeResult="failure" output="Get \"https://10.132.0.40:6567/health\": dial tcp 10.132.0.40:6567: connect: connection refused" Sep 09 07:04:08.276670 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:08.276613 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" podUID="b4f53b83-9137-4998-b3b2-bea46e694595" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.41:6571: connect: connection refused" Sep 09 07:04:08.276670 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:08.276681 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" podUID="b4f53b83-9137-4998-b3b2-bea46e694595" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.41:8443: connect: connection refused" Sep 09 07:04:08.277301 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:08.276637 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" podUID="b4f53b83-9137-4998-b3b2-bea46e694595" containerName="online" probeResult="failure" output="Get \"https://10.132.0.41:6567/health\": dial tcp 10.132.0.41:6567: connect: connection refused" Sep 09 07:04:08.277301 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:08.277166 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" podUID="b4f53b83-9137-4998-b3b2-bea46e694595" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.41:8816: connect: connection refused" Sep 09 07:04:10.276386 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:10.276348 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:04:10.276804 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:10.276404 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:04:10.276804 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:10.276417 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:04:10.276804 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:10.276430 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:04:11.278010 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:11.277966 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:04:11.278508 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:11.278292 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:04:11.278508 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:11.278460 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:04:11.283283 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:11.283255 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:04:11.569323 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:11.569229 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:04:11.569323 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:11.569300 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:04:11.569544 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:11.569390 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:04:11.572071 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:11.572043 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:04:16.378165 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:16.378109 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" podUID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.40:8443: connect: connection refused" Sep 09 07:04:16.378705 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:16.378317 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:04:17.168603 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:17.168554 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" podUID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerName="online" probeResult="failure" output="Get \"https://10.132.0.40:6567/health\": dial tcp 10.132.0.40:6567: connect: connection refused" Sep 09 07:04:17.168998 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:17.168976 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:04:19.544218 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.544190 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:04:19.576706 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.576678 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/54aa0daf-e6f1-408d-b78d-3f4110d28df5-ui-tls\") pod \"54aa0daf-e6f1-408d-b78d-3f4110d28df5\" (UID: \"54aa0daf-e6f1-408d-b78d-3f4110d28df5\") " Sep 09 07:04:19.576706 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.576717 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/54aa0daf-e6f1-408d-b78d-3f4110d28df5-feast-data\") pod \"54aa0daf-e6f1-408d-b78d-3f4110d28df5\" (UID: \"54aa0daf-e6f1-408d-b78d-3f4110d28df5\") " Sep 09 07:04:19.576933 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.576769 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/54aa0daf-e6f1-408d-b78d-3f4110d28df5-online-tls\") pod \"54aa0daf-e6f1-408d-b78d-3f4110d28df5\" (UID: \"54aa0daf-e6f1-408d-b78d-3f4110d28df5\") " Sep 09 07:04:19.576933 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.576799 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6s7z\" (UniqueName: \"kubernetes.io/projected/54aa0daf-e6f1-408d-b78d-3f4110d28df5-kube-api-access-b6s7z\") pod \"54aa0daf-e6f1-408d-b78d-3f4110d28df5\" (UID: \"54aa0daf-e6f1-408d-b78d-3f4110d28df5\") " Sep 09 07:04:19.576933 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.576835 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/54aa0daf-e6f1-408d-b78d-3f4110d28df5-registry-tls\") pod \"54aa0daf-e6f1-408d-b78d-3f4110d28df5\" (UID: \"54aa0daf-e6f1-408d-b78d-3f4110d28df5\") " Sep 09 07:04:19.576933 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.576863 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/54aa0daf-e6f1-408d-b78d-3f4110d28df5-offline-tls\") pod \"54aa0daf-e6f1-408d-b78d-3f4110d28df5\" (UID: \"54aa0daf-e6f1-408d-b78d-3f4110d28df5\") " Sep 09 07:04:19.577710 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.577669 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54aa0daf-e6f1-408d-b78d-3f4110d28df5-feast-data" (OuterVolumeSpecName: "feast-data") pod "54aa0daf-e6f1-408d-b78d-3f4110d28df5" (UID: "54aa0daf-e6f1-408d-b78d-3f4110d28df5"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 07:04:19.579581 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.579554 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54aa0daf-e6f1-408d-b78d-3f4110d28df5-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "54aa0daf-e6f1-408d-b78d-3f4110d28df5" (UID: "54aa0daf-e6f1-408d-b78d-3f4110d28df5"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 07:04:19.579720 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.579694 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54aa0daf-e6f1-408d-b78d-3f4110d28df5-kube-api-access-b6s7z" (OuterVolumeSpecName: "kube-api-access-b6s7z") pod "54aa0daf-e6f1-408d-b78d-3f4110d28df5" (UID: "54aa0daf-e6f1-408d-b78d-3f4110d28df5"). InnerVolumeSpecName "kube-api-access-b6s7z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 07:04:19.580000 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.579970 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54aa0daf-e6f1-408d-b78d-3f4110d28df5-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "54aa0daf-e6f1-408d-b78d-3f4110d28df5" (UID: "54aa0daf-e6f1-408d-b78d-3f4110d28df5"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 07:04:19.580105 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.579978 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54aa0daf-e6f1-408d-b78d-3f4110d28df5-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "54aa0daf-e6f1-408d-b78d-3f4110d28df5" (UID: "54aa0daf-e6f1-408d-b78d-3f4110d28df5"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 07:04:19.580389 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.580366 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54aa0daf-e6f1-408d-b78d-3f4110d28df5-online-tls" (OuterVolumeSpecName: "online-tls") pod "54aa0daf-e6f1-408d-b78d-3f4110d28df5" (UID: "54aa0daf-e6f1-408d-b78d-3f4110d28df5"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 07:04:19.608759 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.608734 2559 generic.go:356] "Generic (PLEG): container finished" podID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerID="dcfacab6ccf95bf050957b23a4258bc51a6f85a2fca721daef8177428b6defda" exitCode=137 Sep 09 07:04:19.608759 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.608756 2559 generic.go:356] "Generic (PLEG): container finished" podID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerID="66f65bfb0a9bb827b80f220d1fdc2be858effdb934dcce279daf7641999fe085" exitCode=137 Sep 09 07:04:19.608906 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.608805 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" Sep 09 07:04:19.608906 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.608821 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" event={"ID":"54aa0daf-e6f1-408d-b78d-3f4110d28df5","Type":"ContainerDied","Data":"dcfacab6ccf95bf050957b23a4258bc51a6f85a2fca721daef8177428b6defda"} Sep 09 07:04:19.608906 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.608873 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" event={"ID":"54aa0daf-e6f1-408d-b78d-3f4110d28df5","Type":"ContainerDied","Data":"66f65bfb0a9bb827b80f220d1fdc2be858effdb934dcce279daf7641999fe085"} Sep 09 07:04:19.608906 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.608893 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld" event={"ID":"54aa0daf-e6f1-408d-b78d-3f4110d28df5","Type":"ContainerDied","Data":"2adf4304f28fefaf5a59581420f9835fb21aac9892611fdd8f8627e3cbf6ca39"} Sep 09 07:04:19.609037 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.608918 2559 scope.go:117] "RemoveContainer" containerID="31c2b88519eae736d24de98ce67228b77729f4b6fd9c21308106edd505d7174f" Sep 09 07:04:19.617473 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.617458 2559 scope.go:117] "RemoveContainer" containerID="dcfacab6ccf95bf050957b23a4258bc51a6f85a2fca721daef8177428b6defda" Sep 09 07:04:19.625001 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.624982 2559 scope.go:117] "RemoveContainer" containerID="9b132fec88521093e07a4c77954a4b935c065aa0fa3d1adc0f69a9c573ac4182" Sep 09 07:04:19.631228 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.631207 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld"] Sep 09 07:04:19.632644 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.632624 2559 scope.go:117] "RemoveContainer" containerID="66f65bfb0a9bb827b80f220d1fdc2be858effdb934dcce279daf7641999fe085" Sep 09 07:04:19.635665 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.635642 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-4vjld"] Sep 09 07:04:19.640356 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.640338 2559 scope.go:117] "RemoveContainer" containerID="d04fd9a71e218cbe6f0ffa469961062aa7775210a3fcf26fe2ccd9d8c87cce94" Sep 09 07:04:19.647610 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.647587 2559 scope.go:117] "RemoveContainer" containerID="3328f0764fe6f1bb38c2d30f0659f271fc7cea715d7ceba208f87b66d880a9f0" Sep 09 07:04:19.654487 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.654470 2559 scope.go:117] "RemoveContainer" containerID="31c2b88519eae736d24de98ce67228b77729f4b6fd9c21308106edd505d7174f" Sep 09 07:04:19.654732 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:04:19.654715 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31c2b88519eae736d24de98ce67228b77729f4b6fd9c21308106edd505d7174f\": container with ID starting with 31c2b88519eae736d24de98ce67228b77729f4b6fd9c21308106edd505d7174f not found: ID does not exist" containerID="31c2b88519eae736d24de98ce67228b77729f4b6fd9c21308106edd505d7174f" Sep 09 07:04:19.654784 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.654738 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31c2b88519eae736d24de98ce67228b77729f4b6fd9c21308106edd505d7174f"} err="failed to get container status \"31c2b88519eae736d24de98ce67228b77729f4b6fd9c21308106edd505d7174f\": rpc error: code = NotFound desc = could not find container \"31c2b88519eae736d24de98ce67228b77729f4b6fd9c21308106edd505d7174f\": container with ID starting with 31c2b88519eae736d24de98ce67228b77729f4b6fd9c21308106edd505d7174f not found: ID does not exist" Sep 09 07:04:19.654784 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.654756 2559 scope.go:117] "RemoveContainer" containerID="dcfacab6ccf95bf050957b23a4258bc51a6f85a2fca721daef8177428b6defda" Sep 09 07:04:19.654984 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:04:19.654969 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcfacab6ccf95bf050957b23a4258bc51a6f85a2fca721daef8177428b6defda\": container with ID starting with dcfacab6ccf95bf050957b23a4258bc51a6f85a2fca721daef8177428b6defda not found: ID does not exist" containerID="dcfacab6ccf95bf050957b23a4258bc51a6f85a2fca721daef8177428b6defda" Sep 09 07:04:19.655026 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.654987 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcfacab6ccf95bf050957b23a4258bc51a6f85a2fca721daef8177428b6defda"} err="failed to get container status \"dcfacab6ccf95bf050957b23a4258bc51a6f85a2fca721daef8177428b6defda\": rpc error: code = NotFound desc = could not find container \"dcfacab6ccf95bf050957b23a4258bc51a6f85a2fca721daef8177428b6defda\": container with ID starting with dcfacab6ccf95bf050957b23a4258bc51a6f85a2fca721daef8177428b6defda not found: ID does not exist" Sep 09 07:04:19.655026 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.655000 2559 scope.go:117] "RemoveContainer" containerID="9b132fec88521093e07a4c77954a4b935c065aa0fa3d1adc0f69a9c573ac4182" Sep 09 07:04:19.655318 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:04:19.655296 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b132fec88521093e07a4c77954a4b935c065aa0fa3d1adc0f69a9c573ac4182\": container with ID starting with 9b132fec88521093e07a4c77954a4b935c065aa0fa3d1adc0f69a9c573ac4182 not found: ID does not exist" containerID="9b132fec88521093e07a4c77954a4b935c065aa0fa3d1adc0f69a9c573ac4182" Sep 09 07:04:19.655391 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.655322 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b132fec88521093e07a4c77954a4b935c065aa0fa3d1adc0f69a9c573ac4182"} err="failed to get container status \"9b132fec88521093e07a4c77954a4b935c065aa0fa3d1adc0f69a9c573ac4182\": rpc error: code = NotFound desc = could not find container \"9b132fec88521093e07a4c77954a4b935c065aa0fa3d1adc0f69a9c573ac4182\": container with ID starting with 9b132fec88521093e07a4c77954a4b935c065aa0fa3d1adc0f69a9c573ac4182 not found: ID does not exist" Sep 09 07:04:19.655391 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.655338 2559 scope.go:117] "RemoveContainer" containerID="66f65bfb0a9bb827b80f220d1fdc2be858effdb934dcce279daf7641999fe085" Sep 09 07:04:19.655578 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:04:19.655563 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66f65bfb0a9bb827b80f220d1fdc2be858effdb934dcce279daf7641999fe085\": container with ID starting with 66f65bfb0a9bb827b80f220d1fdc2be858effdb934dcce279daf7641999fe085 not found: ID does not exist" containerID="66f65bfb0a9bb827b80f220d1fdc2be858effdb934dcce279daf7641999fe085" Sep 09 07:04:19.655623 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.655589 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66f65bfb0a9bb827b80f220d1fdc2be858effdb934dcce279daf7641999fe085"} err="failed to get container status \"66f65bfb0a9bb827b80f220d1fdc2be858effdb934dcce279daf7641999fe085\": rpc error: code = NotFound desc = could not find container \"66f65bfb0a9bb827b80f220d1fdc2be858effdb934dcce279daf7641999fe085\": container with ID starting with 66f65bfb0a9bb827b80f220d1fdc2be858effdb934dcce279daf7641999fe085 not found: ID does not exist" Sep 09 07:04:19.655623 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.655602 2559 scope.go:117] "RemoveContainer" containerID="d04fd9a71e218cbe6f0ffa469961062aa7775210a3fcf26fe2ccd9d8c87cce94" Sep 09 07:04:19.655853 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:04:19.655834 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d04fd9a71e218cbe6f0ffa469961062aa7775210a3fcf26fe2ccd9d8c87cce94\": container with ID starting with d04fd9a71e218cbe6f0ffa469961062aa7775210a3fcf26fe2ccd9d8c87cce94 not found: ID does not exist" containerID="d04fd9a71e218cbe6f0ffa469961062aa7775210a3fcf26fe2ccd9d8c87cce94" Sep 09 07:04:19.655920 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.655861 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d04fd9a71e218cbe6f0ffa469961062aa7775210a3fcf26fe2ccd9d8c87cce94"} err="failed to get container status \"d04fd9a71e218cbe6f0ffa469961062aa7775210a3fcf26fe2ccd9d8c87cce94\": rpc error: code = NotFound desc = could not find container \"d04fd9a71e218cbe6f0ffa469961062aa7775210a3fcf26fe2ccd9d8c87cce94\": container with ID starting with d04fd9a71e218cbe6f0ffa469961062aa7775210a3fcf26fe2ccd9d8c87cce94 not found: ID does not exist" Sep 09 07:04:19.655920 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.655881 2559 scope.go:117] "RemoveContainer" containerID="3328f0764fe6f1bb38c2d30f0659f271fc7cea715d7ceba208f87b66d880a9f0" Sep 09 07:04:19.656152 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:04:19.656135 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3328f0764fe6f1bb38c2d30f0659f271fc7cea715d7ceba208f87b66d880a9f0\": container with ID starting with 3328f0764fe6f1bb38c2d30f0659f271fc7cea715d7ceba208f87b66d880a9f0 not found: ID does not exist" containerID="3328f0764fe6f1bb38c2d30f0659f271fc7cea715d7ceba208f87b66d880a9f0" Sep 09 07:04:19.656203 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.656170 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3328f0764fe6f1bb38c2d30f0659f271fc7cea715d7ceba208f87b66d880a9f0"} err="failed to get container status \"3328f0764fe6f1bb38c2d30f0659f271fc7cea715d7ceba208f87b66d880a9f0\": rpc error: code = NotFound desc = could not find container \"3328f0764fe6f1bb38c2d30f0659f271fc7cea715d7ceba208f87b66d880a9f0\": container with ID starting with 3328f0764fe6f1bb38c2d30f0659f271fc7cea715d7ceba208f87b66d880a9f0 not found: ID does not exist" Sep 09 07:04:19.656203 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.656184 2559 scope.go:117] "RemoveContainer" containerID="31c2b88519eae736d24de98ce67228b77729f4b6fd9c21308106edd505d7174f" Sep 09 07:04:19.656425 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.656406 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31c2b88519eae736d24de98ce67228b77729f4b6fd9c21308106edd505d7174f"} err="failed to get container status \"31c2b88519eae736d24de98ce67228b77729f4b6fd9c21308106edd505d7174f\": rpc error: code = NotFound desc = could not find container \"31c2b88519eae736d24de98ce67228b77729f4b6fd9c21308106edd505d7174f\": container with ID starting with 31c2b88519eae736d24de98ce67228b77729f4b6fd9c21308106edd505d7174f not found: ID does not exist" Sep 09 07:04:19.656470 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.656426 2559 scope.go:117] "RemoveContainer" containerID="dcfacab6ccf95bf050957b23a4258bc51a6f85a2fca721daef8177428b6defda" Sep 09 07:04:19.656657 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.656638 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcfacab6ccf95bf050957b23a4258bc51a6f85a2fca721daef8177428b6defda"} err="failed to get container status \"dcfacab6ccf95bf050957b23a4258bc51a6f85a2fca721daef8177428b6defda\": rpc error: code = NotFound desc = could not find container \"dcfacab6ccf95bf050957b23a4258bc51a6f85a2fca721daef8177428b6defda\": container with ID starting with dcfacab6ccf95bf050957b23a4258bc51a6f85a2fca721daef8177428b6defda not found: ID does not exist" Sep 09 07:04:19.656725 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.656659 2559 scope.go:117] "RemoveContainer" containerID="9b132fec88521093e07a4c77954a4b935c065aa0fa3d1adc0f69a9c573ac4182" Sep 09 07:04:19.656918 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.656900 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b132fec88521093e07a4c77954a4b935c065aa0fa3d1adc0f69a9c573ac4182"} err="failed to get container status \"9b132fec88521093e07a4c77954a4b935c065aa0fa3d1adc0f69a9c573ac4182\": rpc error: code = NotFound desc = could not find container \"9b132fec88521093e07a4c77954a4b935c065aa0fa3d1adc0f69a9c573ac4182\": container with ID starting with 9b132fec88521093e07a4c77954a4b935c065aa0fa3d1adc0f69a9c573ac4182 not found: ID does not exist" Sep 09 07:04:19.656966 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.656919 2559 scope.go:117] "RemoveContainer" containerID="66f65bfb0a9bb827b80f220d1fdc2be858effdb934dcce279daf7641999fe085" Sep 09 07:04:19.657180 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.657158 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66f65bfb0a9bb827b80f220d1fdc2be858effdb934dcce279daf7641999fe085"} err="failed to get container status \"66f65bfb0a9bb827b80f220d1fdc2be858effdb934dcce279daf7641999fe085\": rpc error: code = NotFound desc = could not find container \"66f65bfb0a9bb827b80f220d1fdc2be858effdb934dcce279daf7641999fe085\": container with ID starting with 66f65bfb0a9bb827b80f220d1fdc2be858effdb934dcce279daf7641999fe085 not found: ID does not exist" Sep 09 07:04:19.657259 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.657181 2559 scope.go:117] "RemoveContainer" containerID="d04fd9a71e218cbe6f0ffa469961062aa7775210a3fcf26fe2ccd9d8c87cce94" Sep 09 07:04:19.657442 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.657422 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d04fd9a71e218cbe6f0ffa469961062aa7775210a3fcf26fe2ccd9d8c87cce94"} err="failed to get container status \"d04fd9a71e218cbe6f0ffa469961062aa7775210a3fcf26fe2ccd9d8c87cce94\": rpc error: code = NotFound desc = could not find container \"d04fd9a71e218cbe6f0ffa469961062aa7775210a3fcf26fe2ccd9d8c87cce94\": container with ID starting with d04fd9a71e218cbe6f0ffa469961062aa7775210a3fcf26fe2ccd9d8c87cce94 not found: ID does not exist" Sep 09 07:04:19.657484 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.657444 2559 scope.go:117] "RemoveContainer" containerID="3328f0764fe6f1bb38c2d30f0659f271fc7cea715d7ceba208f87b66d880a9f0" Sep 09 07:04:19.657680 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.657658 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3328f0764fe6f1bb38c2d30f0659f271fc7cea715d7ceba208f87b66d880a9f0"} err="failed to get container status \"3328f0764fe6f1bb38c2d30f0659f271fc7cea715d7ceba208f87b66d880a9f0\": rpc error: code = NotFound desc = could not find container \"3328f0764fe6f1bb38c2d30f0659f271fc7cea715d7ceba208f87b66d880a9f0\": container with ID starting with 3328f0764fe6f1bb38c2d30f0659f271fc7cea715d7ceba208f87b66d880a9f0 not found: ID does not exist" Sep 09 07:04:19.678267 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.678244 2559 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/54aa0daf-e6f1-408d-b78d-3f4110d28df5-online-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:04:19.678267 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.678264 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6s7z\" (UniqueName: \"kubernetes.io/projected/54aa0daf-e6f1-408d-b78d-3f4110d28df5-kube-api-access-b6s7z\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:04:19.678369 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.678273 2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/54aa0daf-e6f1-408d-b78d-3f4110d28df5-registry-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:04:19.678369 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.678282 2559 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/54aa0daf-e6f1-408d-b78d-3f4110d28df5-offline-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:04:19.678369 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.678289 2559 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/54aa0daf-e6f1-408d-b78d-3f4110d28df5-ui-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:04:19.678369 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:19.678296 2559 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/54aa0daf-e6f1-408d-b78d-3f4110d28df5-feast-data\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:04:20.701837 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:04:20.701806 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" path="/var/lib/kubelet/pods/54aa0daf-e6f1-408d-b78d-3f4110d28df5/volumes" Sep 09 07:05:14.232705 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.232671 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4"] Sep 09 07:05:14.233221 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.233039 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerName="ui" Sep 09 07:05:14.233221 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.233065 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerName="ui" Sep 09 07:05:14.233221 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.233080 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerName="offline" Sep 09 07:05:14.233221 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.233086 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerName="offline" Sep 09 07:05:14.233221 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.233094 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerName="online" Sep 09 07:05:14.233221 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.233100 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerName="online" Sep 09 07:05:14.233221 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.233112 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerName="registry" Sep 09 07:05:14.233221 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.233117 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerName="registry" Sep 09 07:05:14.233221 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.233125 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerName="feast-apply" Sep 09 07:05:14.233221 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.233130 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerName="feast-apply" Sep 09 07:05:14.233221 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.233137 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerName="feast-init" Sep 09 07:05:14.233221 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.233142 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerName="feast-init" Sep 09 07:05:14.233221 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.233202 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerName="online" Sep 09 07:05:14.233221 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.233211 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerName="ui" Sep 09 07:05:14.233221 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.233217 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerName="offline" Sep 09 07:05:14.233221 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.233225 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="54aa0daf-e6f1-408d-b78d-3f4110d28df5" containerName="registry" Sep 09 07:05:14.236645 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.236628 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" Sep 09 07:05:14.239703 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.239680 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 09 07:05:14.239863 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.239683 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 09 07:05:14.239863 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.239767 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 09 07:05:14.239998 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.239985 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 09 07:05:14.240045 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.240010 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-zpt6s\"" Sep 09 07:05:14.240110 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.239987 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 09 07:05:14.242616 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.242597 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 09 07:05:14.254705 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.254678 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4"] Sep 09 07:05:14.311288 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.311254 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/72aca48e-fd59-4130-b4f5-570a5e0629d3-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-4z6z4\" (UID: \"72aca48e-fd59-4130-b4f5-570a5e0629d3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" Sep 09 07:05:14.311288 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.311293 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/72aca48e-fd59-4130-b4f5-570a5e0629d3-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-4z6z4\" (UID: \"72aca48e-fd59-4130-b4f5-570a5e0629d3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" Sep 09 07:05:14.311484 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.311315 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddzkn\" (UniqueName: \"kubernetes.io/projected/72aca48e-fd59-4130-b4f5-570a5e0629d3-kube-api-access-ddzkn\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-4z6z4\" (UID: \"72aca48e-fd59-4130-b4f5-570a5e0629d3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" Sep 09 07:05:14.311484 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.311357 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/72aca48e-fd59-4130-b4f5-570a5e0629d3-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-4z6z4\" (UID: \"72aca48e-fd59-4130-b4f5-570a5e0629d3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" Sep 09 07:05:14.311484 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.311376 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/72aca48e-fd59-4130-b4f5-570a5e0629d3-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-4z6z4\" (UID: \"72aca48e-fd59-4130-b4f5-570a5e0629d3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" Sep 09 07:05:14.311484 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.311449 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/72aca48e-fd59-4130-b4f5-570a5e0629d3-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-4z6z4\" (UID: \"72aca48e-fd59-4130-b4f5-570a5e0629d3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" Sep 09 07:05:14.412042 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.412008 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/72aca48e-fd59-4130-b4f5-570a5e0629d3-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-4z6z4\" (UID: \"72aca48e-fd59-4130-b4f5-570a5e0629d3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" Sep 09 07:05:14.412243 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.412045 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/72aca48e-fd59-4130-b4f5-570a5e0629d3-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-4z6z4\" (UID: \"72aca48e-fd59-4130-b4f5-570a5e0629d3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" Sep 09 07:05:14.412243 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.412083 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddzkn\" (UniqueName: \"kubernetes.io/projected/72aca48e-fd59-4130-b4f5-570a5e0629d3-kube-api-access-ddzkn\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-4z6z4\" (UID: \"72aca48e-fd59-4130-b4f5-570a5e0629d3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" Sep 09 07:05:14.412243 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.412120 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/72aca48e-fd59-4130-b4f5-570a5e0629d3-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-4z6z4\" (UID: \"72aca48e-fd59-4130-b4f5-570a5e0629d3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" Sep 09 07:05:14.412243 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.412139 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/72aca48e-fd59-4130-b4f5-570a5e0629d3-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-4z6z4\" (UID: \"72aca48e-fd59-4130-b4f5-570a5e0629d3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" Sep 09 07:05:14.412243 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.412173 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/72aca48e-fd59-4130-b4f5-570a5e0629d3-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-4z6z4\" (UID: \"72aca48e-fd59-4130-b4f5-570a5e0629d3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" Sep 09 07:05:14.412529 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:05:14.412282 2559 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-offline-tls: secret "feast-simple-feast-remote-setup-offline-tls" not found Sep 09 07:05:14.412529 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:05:14.412391 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72aca48e-fd59-4130-b4f5-570a5e0629d3-offline-tls podName:72aca48e-fd59-4130-b4f5-570a5e0629d3 nodeName:}" failed. No retries permitted until 2026-09-09 07:05:14.91236968 +0000 UTC m=+1612.808787094 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/72aca48e-fd59-4130-b4f5-570a5e0629d3-offline-tls") pod "feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" (UID: "72aca48e-fd59-4130-b4f5-570a5e0629d3") : secret "feast-simple-feast-remote-setup-offline-tls" not found Sep 09 07:05:14.412615 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.412570 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/72aca48e-fd59-4130-b4f5-570a5e0629d3-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-4z6z4\" (UID: \"72aca48e-fd59-4130-b4f5-570a5e0629d3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" Sep 09 07:05:14.412870 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.412850 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/72aca48e-fd59-4130-b4f5-570a5e0629d3-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-4z6z4\" (UID: \"72aca48e-fd59-4130-b4f5-570a5e0629d3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" Sep 09 07:05:14.414487 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.414456 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/72aca48e-fd59-4130-b4f5-570a5e0629d3-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-4z6z4\" (UID: \"72aca48e-fd59-4130-b4f5-570a5e0629d3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" Sep 09 07:05:14.414575 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.414514 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/72aca48e-fd59-4130-b4f5-570a5e0629d3-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-4z6z4\" (UID: \"72aca48e-fd59-4130-b4f5-570a5e0629d3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" Sep 09 07:05:14.424135 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.424114 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddzkn\" (UniqueName: \"kubernetes.io/projected/72aca48e-fd59-4130-b4f5-570a5e0629d3-kube-api-access-ddzkn\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-4z6z4\" (UID: \"72aca48e-fd59-4130-b4f5-570a5e0629d3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" Sep 09 07:05:14.917638 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.917602 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/72aca48e-fd59-4130-b4f5-570a5e0629d3-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-4z6z4\" (UID: \"72aca48e-fd59-4130-b4f5-570a5e0629d3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" Sep 09 07:05:14.920001 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:14.919981 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/72aca48e-fd59-4130-b4f5-570a5e0629d3-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-4z6z4\" (UID: \"72aca48e-fd59-4130-b4f5-570a5e0629d3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" Sep 09 07:05:15.146020 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:15.145941 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" Sep 09 07:05:15.425457 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:15.425430 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4"] Sep 09 07:05:15.427805 ip-10-0-132-137 kubenswrapper[2559]: W0909 07:05:15.427778 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod72aca48e_fd59_4130_b4f5_570a5e0629d3.slice/crio-a33577ef774127f5e47e9ff5272b3cb9be8321e1c56ca5c1277deb945524d3d5 WatchSource:0}: Error finding container a33577ef774127f5e47e9ff5272b3cb9be8321e1c56ca5c1277deb945524d3d5: Status 404 returned error can't find the container with id a33577ef774127f5e47e9ff5272b3cb9be8321e1c56ca5c1277deb945524d3d5 Sep 09 07:05:15.844695 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:15.844594 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" event={"ID":"72aca48e-fd59-4130-b4f5-570a5e0629d3","Type":"ContainerStarted","Data":"c011de24d22196e9de5cf4d803b4ddbc6a03e064ae63b2542a54ae2f5132a1ac"} Sep 09 07:05:15.844695 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:15.844648 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" event={"ID":"72aca48e-fd59-4130-b4f5-570a5e0629d3","Type":"ContainerStarted","Data":"a33577ef774127f5e47e9ff5272b3cb9be8321e1c56ca5c1277deb945524d3d5"} Sep 09 07:05:20.868446 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:20.868410 2559 generic.go:356] "Generic (PLEG): container finished" podID="72aca48e-fd59-4130-b4f5-570a5e0629d3" containerID="c011de24d22196e9de5cf4d803b4ddbc6a03e064ae63b2542a54ae2f5132a1ac" exitCode=0 Sep 09 07:05:20.868882 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:20.868481 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" event={"ID":"72aca48e-fd59-4130-b4f5-570a5e0629d3","Type":"ContainerDied","Data":"c011de24d22196e9de5cf4d803b4ddbc6a03e064ae63b2542a54ae2f5132a1ac"} Sep 09 07:05:21.874377 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:21.874336 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" event={"ID":"72aca48e-fd59-4130-b4f5-570a5e0629d3","Type":"ContainerStarted","Data":"6a6a9eab55b65eea2eda481386b2b493f8fc9f1f0660004f296dd1a511e3ae8a"} Sep 09 07:05:49.996021 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:49.995937 2559 generic.go:356] "Generic (PLEG): container finished" podID="72aca48e-fd59-4130-b4f5-570a5e0629d3" containerID="6a6a9eab55b65eea2eda481386b2b493f8fc9f1f0660004f296dd1a511e3ae8a" exitCode=0 Sep 09 07:05:49.996387 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:49.996015 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" event={"ID":"72aca48e-fd59-4130-b4f5-570a5e0629d3","Type":"ContainerDied","Data":"6a6a9eab55b65eea2eda481386b2b493f8fc9f1f0660004f296dd1a511e3ae8a"} Sep 09 07:05:51.006394 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:51.006351 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" event={"ID":"72aca48e-fd59-4130-b4f5-570a5e0629d3","Type":"ContainerStarted","Data":"125225ed79b941ceb8926796013778ac4ca16802594f2ee95d968a94ee4fc479"} Sep 09 07:05:51.006853 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:51.006426 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" event={"ID":"72aca48e-fd59-4130-b4f5-570a5e0629d3","Type":"ContainerStarted","Data":"b67c731c0e4ac190f69200fd5c2640445541d9fb4131a1912935ac82382f12ef"} Sep 09 07:05:51.006853 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:51.006448 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" event={"ID":"72aca48e-fd59-4130-b4f5-570a5e0629d3","Type":"ContainerStarted","Data":"1cae1ad2e337fde2d70d17eba944bee8a03ec616b938356c8ad4317d1b196df4"} Sep 09 07:05:51.044215 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:51.044159 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" podStartSLOduration=37.044138327 podStartE2EDuration="37.044138327s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:05:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:05:51.042739238 +0000 UTC m=+1648.939156662" watchObservedRunningTime="2026-09-09 07:05:51.044138327 +0000 UTC m=+1648.940555750" Sep 09 07:05:51.146297 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:51.146258 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" Sep 09 07:05:51.146471 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:51.146322 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" Sep 09 07:05:51.146471 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:51.146345 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" Sep 09 07:05:51.148296 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:51.148254 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" podUID="72aca48e-fd59-4130-b4f5-570a5e0629d3" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.42:8443: connect: connection refused" Sep 09 07:05:51.148457 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:51.148299 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" podUID="72aca48e-fd59-4130-b4f5-570a5e0629d3" containerName="online" probeResult="failure" output="Get \"https://10.132.0.42:6567/health\": dial tcp 10.132.0.42:6567: connect: connection refused" Sep 09 07:05:51.148457 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:51.148292 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" podUID="72aca48e-fd59-4130-b4f5-570a5e0629d3" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.42:8816: connect: connection refused" Sep 09 07:05:54.146275 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:54.146226 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" podUID="72aca48e-fd59-4130-b4f5-570a5e0629d3" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.42:8816: connect: connection refused" Sep 09 07:05:54.146275 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:54.146247 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" podUID="72aca48e-fd59-4130-b4f5-570a5e0629d3" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.42:8443: connect: connection refused" Sep 09 07:05:54.146928 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:54.146281 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" podUID="72aca48e-fd59-4130-b4f5-570a5e0629d3" containerName="online" probeResult="failure" output="Get \"https://10.132.0.42:6567/health\": dial tcp 10.132.0.42:6567: connect: connection refused" Sep 09 07:05:55.147083 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:55.147030 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" Sep 09 07:05:55.147470 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:55.147111 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" Sep 09 07:05:55.147470 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:55.147137 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" Sep 09 07:05:57.146427 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:57.146383 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" podUID="72aca48e-fd59-4130-b4f5-570a5e0629d3" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.42:8816: connect: connection refused" Sep 09 07:05:57.146427 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:57.146411 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" podUID="72aca48e-fd59-4130-b4f5-570a5e0629d3" containerName="online" probeResult="failure" output="Get \"https://10.132.0.42:6567/health\": dial tcp 10.132.0.42:6567: connect: connection refused" Sep 09 07:05:57.146914 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:05:57.146382 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" podUID="72aca48e-fd59-4130-b4f5-570a5e0629d3" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.42:8443: connect: connection refused" Sep 09 07:06:00.148122 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:06:00.148085 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" Sep 09 07:06:00.148550 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:06:00.148231 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" Sep 09 07:06:00.148900 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:06:00.148874 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" Sep 09 07:06:00.149216 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:06:00.149196 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" Sep 09 07:06:00.852412 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:06:00.852379 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" Sep 09 07:06:01.784207 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:06:01.784169 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" Sep 09 07:07:04.663094 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:04.663000 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4"] Sep 09 07:07:04.663595 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:04.663300 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" podUID="72aca48e-fd59-4130-b4f5-570a5e0629d3" containerName="online" containerID="cri-o://1cae1ad2e337fde2d70d17eba944bee8a03ec616b938356c8ad4317d1b196df4" gracePeriod=30 Sep 09 07:07:04.663595 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:04.663379 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" podUID="72aca48e-fd59-4130-b4f5-570a5e0629d3" containerName="offline" containerID="cri-o://b67c731c0e4ac190f69200fd5c2640445541d9fb4131a1912935ac82382f12ef" gracePeriod=30 Sep 09 07:07:04.663595 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:04.663393 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" podUID="72aca48e-fd59-4130-b4f5-570a5e0629d3" containerName="ui" containerID="cri-o://125225ed79b941ceb8926796013778ac4ca16802594f2ee95d968a94ee4fc479" gracePeriod=30 Sep 09 07:07:04.838037 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:04.837997 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx"] Sep 09 07:07:04.838369 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:04.838316 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" podUID="b4f53b83-9137-4998-b3b2-bea46e694595" containerName="registry" containerID="cri-o://e2568c0847b7d49e0b4d754bce7d196caff0ff03feaa4684878a1c67e8659469" gracePeriod=30 Sep 09 07:07:04.838487 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:04.838408 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" podUID="b4f53b83-9137-4998-b3b2-bea46e694595" containerName="online" containerID="cri-o://6bc5cd7d29647903ebbb1d24b4c50aaa93b3c8d6c224b93b75d5310d256fcd0c" gracePeriod=30 Sep 09 07:07:04.838556 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:04.838492 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" podUID="b4f53b83-9137-4998-b3b2-bea46e694595" containerName="ui" containerID="cri-o://16df504799e061bdbce19d9f30879c994b26ee7198bac76dd0d88c4e057d9897" gracePeriod=30 Sep 09 07:07:04.838684 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:04.838654 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" podUID="b4f53b83-9137-4998-b3b2-bea46e694595" containerName="offline" containerID="cri-o://98d4b97907afa776bdd9d6ae7ea642008a953f94f1b4631192d1fcb783fb30f2" gracePeriod=30 Sep 09 07:07:06.333616 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:06.333581 2559 generic.go:356] "Generic (PLEG): container finished" podID="b4f53b83-9137-4998-b3b2-bea46e694595" containerID="16df504799e061bdbce19d9f30879c994b26ee7198bac76dd0d88c4e057d9897" exitCode=0 Sep 09 07:07:06.333616 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:06.333609 2559 generic.go:356] "Generic (PLEG): container finished" podID="b4f53b83-9137-4998-b3b2-bea46e694595" containerID="6bc5cd7d29647903ebbb1d24b4c50aaa93b3c8d6c224b93b75d5310d256fcd0c" exitCode=0 Sep 09 07:07:06.334126 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:06.333652 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" event={"ID":"b4f53b83-9137-4998-b3b2-bea46e694595","Type":"ContainerDied","Data":"16df504799e061bdbce19d9f30879c994b26ee7198bac76dd0d88c4e057d9897"} Sep 09 07:07:06.334126 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:06.333697 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" event={"ID":"b4f53b83-9137-4998-b3b2-bea46e694595","Type":"ContainerDied","Data":"6bc5cd7d29647903ebbb1d24b4c50aaa93b3c8d6c224b93b75d5310d256fcd0c"} Sep 09 07:07:06.336243 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:06.336217 2559 generic.go:356] "Generic (PLEG): container finished" podID="72aca48e-fd59-4130-b4f5-570a5e0629d3" containerID="125225ed79b941ceb8926796013778ac4ca16802594f2ee95d968a94ee4fc479" exitCode=0 Sep 09 07:07:06.336243 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:06.336241 2559 generic.go:356] "Generic (PLEG): container finished" podID="72aca48e-fd59-4130-b4f5-570a5e0629d3" containerID="1cae1ad2e337fde2d70d17eba944bee8a03ec616b938356c8ad4317d1b196df4" exitCode=0 Sep 09 07:07:06.336392 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:06.336284 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" event={"ID":"72aca48e-fd59-4130-b4f5-570a5e0629d3","Type":"ContainerDied","Data":"125225ed79b941ceb8926796013778ac4ca16802594f2ee95d968a94ee4fc479"} Sep 09 07:07:06.336392 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:06.336319 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" event={"ID":"72aca48e-fd59-4130-b4f5-570a5e0629d3","Type":"ContainerDied","Data":"1cae1ad2e337fde2d70d17eba944bee8a03ec616b938356c8ad4317d1b196df4"} Sep 09 07:07:10.149018 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:10.148967 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" podUID="72aca48e-fd59-4130-b4f5-570a5e0629d3" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.42:8443: connect: connection refused" Sep 09 07:07:11.057364 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:11.057323 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" podUID="72aca48e-fd59-4130-b4f5-570a5e0629d3" containerName="online" probeResult="failure" output="Get \"https://10.132.0.42:6567/health\": dial tcp 10.132.0.42:6567: connect: connection refused" Sep 09 07:07:11.569861 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:11.569812 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" podUID="b4f53b83-9137-4998-b3b2-bea46e694595" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.41:8443: connect: connection refused" Sep 09 07:07:11.569861 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:11.569837 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" podUID="b4f53b83-9137-4998-b3b2-bea46e694595" containerName="online" probeResult="failure" output="Get \"https://10.132.0.41:6567/health\": dial tcp 10.132.0.41:6567: connect: connection refused" Sep 09 07:07:20.149366 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:20.149319 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" podUID="72aca48e-fd59-4130-b4f5-570a5e0629d3" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.42:8443: connect: connection refused" Sep 09 07:07:21.057027 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:21.056979 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" podUID="72aca48e-fd59-4130-b4f5-570a5e0629d3" containerName="online" probeResult="failure" output="Get \"https://10.132.0.42:6567/health\": dial tcp 10.132.0.42:6567: connect: connection refused" Sep 09 07:07:21.569541 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:21.569498 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" podUID="b4f53b83-9137-4998-b3b2-bea46e694595" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.41:8443: connect: connection refused" Sep 09 07:07:21.570015 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:21.569585 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" podUID="b4f53b83-9137-4998-b3b2-bea46e694595" containerName="online" probeResult="failure" output="Get \"https://10.132.0.41:6567/health\": dial tcp 10.132.0.41:6567: connect: connection refused" Sep 09 07:07:30.149513 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:30.149455 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" podUID="72aca48e-fd59-4130-b4f5-570a5e0629d3" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.42:8443: connect: connection refused" Sep 09 07:07:30.149994 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:30.149604 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" Sep 09 07:07:31.057817 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:31.057778 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" podUID="72aca48e-fd59-4130-b4f5-570a5e0629d3" containerName="online" probeResult="failure" output="Get \"https://10.132.0.42:6567/health\": dial tcp 10.132.0.42:6567: connect: connection refused" Sep 09 07:07:31.058010 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:31.057885 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" Sep 09 07:07:31.569105 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:31.569039 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" podUID="b4f53b83-9137-4998-b3b2-bea46e694595" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.41:8443: connect: connection refused" Sep 09 07:07:31.569105 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:31.569090 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" podUID="b4f53b83-9137-4998-b3b2-bea46e694595" containerName="online" probeResult="failure" output="Get \"https://10.132.0.41:6567/health\": dial tcp 10.132.0.41:6567: connect: connection refused" Sep 09 07:07:31.569609 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:31.569201 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:07:31.569609 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:31.569264 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:07:34.833922 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:34.833899 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" Sep 09 07:07:34.918431 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:34.918356 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ddzkn\" (UniqueName: \"kubernetes.io/projected/72aca48e-fd59-4130-b4f5-570a5e0629d3-kube-api-access-ddzkn\") pod \"72aca48e-fd59-4130-b4f5-570a5e0629d3\" (UID: \"72aca48e-fd59-4130-b4f5-570a5e0629d3\") " Sep 09 07:07:34.918431 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:34.918397 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/72aca48e-fd59-4130-b4f5-570a5e0629d3-registry-tls\") pod \"72aca48e-fd59-4130-b4f5-570a5e0629d3\" (UID: \"72aca48e-fd59-4130-b4f5-570a5e0629d3\") " Sep 09 07:07:34.918431 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:34.918429 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/72aca48e-fd59-4130-b4f5-570a5e0629d3-offline-tls\") pod \"72aca48e-fd59-4130-b4f5-570a5e0629d3\" (UID: \"72aca48e-fd59-4130-b4f5-570a5e0629d3\") " Sep 09 07:07:34.918698 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:34.918451 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/72aca48e-fd59-4130-b4f5-570a5e0629d3-online-tls\") pod \"72aca48e-fd59-4130-b4f5-570a5e0629d3\" (UID: \"72aca48e-fd59-4130-b4f5-570a5e0629d3\") " Sep 09 07:07:34.918760 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:34.918692 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/72aca48e-fd59-4130-b4f5-570a5e0629d3-feast-data\") pod \"72aca48e-fd59-4130-b4f5-570a5e0629d3\" (UID: \"72aca48e-fd59-4130-b4f5-570a5e0629d3\") " Sep 09 07:07:34.918760 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:34.918750 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/72aca48e-fd59-4130-b4f5-570a5e0629d3-ui-tls\") pod \"72aca48e-fd59-4130-b4f5-570a5e0629d3\" (UID: \"72aca48e-fd59-4130-b4f5-570a5e0629d3\") " Sep 09 07:07:34.918879 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:34.918803 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72aca48e-fd59-4130-b4f5-570a5e0629d3-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "72aca48e-fd59-4130-b4f5-570a5e0629d3" (UID: "72aca48e-fd59-4130-b4f5-570a5e0629d3"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 07:07:34.919132 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:34.919086 2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/72aca48e-fd59-4130-b4f5-570a5e0629d3-registry-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:07:34.919573 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:34.919541 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72aca48e-fd59-4130-b4f5-570a5e0629d3-feast-data" (OuterVolumeSpecName: "feast-data") pod "72aca48e-fd59-4130-b4f5-570a5e0629d3" (UID: "72aca48e-fd59-4130-b4f5-570a5e0629d3"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 07:07:34.920835 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:34.920799 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72aca48e-fd59-4130-b4f5-570a5e0629d3-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "72aca48e-fd59-4130-b4f5-570a5e0629d3" (UID: "72aca48e-fd59-4130-b4f5-570a5e0629d3"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 07:07:34.920917 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:34.920834 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72aca48e-fd59-4130-b4f5-570a5e0629d3-kube-api-access-ddzkn" (OuterVolumeSpecName: "kube-api-access-ddzkn") pod "72aca48e-fd59-4130-b4f5-570a5e0629d3" (UID: "72aca48e-fd59-4130-b4f5-570a5e0629d3"). InnerVolumeSpecName "kube-api-access-ddzkn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 07:07:34.920960 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:34.920946 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72aca48e-fd59-4130-b4f5-570a5e0629d3-online-tls" (OuterVolumeSpecName: "online-tls") pod "72aca48e-fd59-4130-b4f5-570a5e0629d3" (UID: "72aca48e-fd59-4130-b4f5-570a5e0629d3"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 07:07:34.921192 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:34.921031 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72aca48e-fd59-4130-b4f5-570a5e0629d3-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "72aca48e-fd59-4130-b4f5-570a5e0629d3" (UID: "72aca48e-fd59-4130-b4f5-570a5e0629d3"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 07:07:35.020467 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.020438 2559 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/72aca48e-fd59-4130-b4f5-570a5e0629d3-feast-data\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:07:35.020467 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.020463 2559 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/72aca48e-fd59-4130-b4f5-570a5e0629d3-ui-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:07:35.020620 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.020473 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddzkn\" (UniqueName: \"kubernetes.io/projected/72aca48e-fd59-4130-b4f5-570a5e0629d3-kube-api-access-ddzkn\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:07:35.020620 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.020483 2559 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/72aca48e-fd59-4130-b4f5-570a5e0629d3-offline-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:07:35.020620 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.020492 2559 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/72aca48e-fd59-4130-b4f5-570a5e0629d3-online-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:07:35.464122 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.464086 2559 generic.go:356] "Generic (PLEG): container finished" podID="72aca48e-fd59-4130-b4f5-570a5e0629d3" containerID="b67c731c0e4ac190f69200fd5c2640445541d9fb4131a1912935ac82382f12ef" exitCode=137 Sep 09 07:07:35.464394 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.464177 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" event={"ID":"72aca48e-fd59-4130-b4f5-570a5e0629d3","Type":"ContainerDied","Data":"b67c731c0e4ac190f69200fd5c2640445541d9fb4131a1912935ac82382f12ef"} Sep 09 07:07:35.464394 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.464210 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" Sep 09 07:07:35.464394 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.464240 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4" event={"ID":"72aca48e-fd59-4130-b4f5-570a5e0629d3","Type":"ContainerDied","Data":"a33577ef774127f5e47e9ff5272b3cb9be8321e1c56ca5c1277deb945524d3d5"} Sep 09 07:07:35.464394 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.464268 2559 scope.go:117] "RemoveContainer" containerID="125225ed79b941ceb8926796013778ac4ca16802594f2ee95d968a94ee4fc479" Sep 09 07:07:35.468264 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.468237 2559 generic.go:356] "Generic (PLEG): container finished" podID="b4f53b83-9137-4998-b3b2-bea46e694595" containerID="98d4b97907afa776bdd9d6ae7ea642008a953f94f1b4631192d1fcb783fb30f2" exitCode=137 Sep 09 07:07:35.468264 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.468262 2559 generic.go:356] "Generic (PLEG): container finished" podID="b4f53b83-9137-4998-b3b2-bea46e694595" containerID="e2568c0847b7d49e0b4d754bce7d196caff0ff03feaa4684878a1c67e8659469" exitCode=137 Sep 09 07:07:35.468423 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.468295 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" event={"ID":"b4f53b83-9137-4998-b3b2-bea46e694595","Type":"ContainerDied","Data":"98d4b97907afa776bdd9d6ae7ea642008a953f94f1b4631192d1fcb783fb30f2"} Sep 09 07:07:35.468423 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.468337 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" event={"ID":"b4f53b83-9137-4998-b3b2-bea46e694595","Type":"ContainerDied","Data":"e2568c0847b7d49e0b4d754bce7d196caff0ff03feaa4684878a1c67e8659469"} Sep 09 07:07:35.474295 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.474276 2559 scope.go:117] "RemoveContainer" containerID="b67c731c0e4ac190f69200fd5c2640445541d9fb4131a1912935ac82382f12ef" Sep 09 07:07:35.482774 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.482757 2559 scope.go:117] "RemoveContainer" containerID="1cae1ad2e337fde2d70d17eba944bee8a03ec616b938356c8ad4317d1b196df4" Sep 09 07:07:35.484766 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.484750 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:07:35.489201 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.489180 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4"] Sep 09 07:07:35.490852 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.490826 2559 scope.go:117] "RemoveContainer" containerID="6a6a9eab55b65eea2eda481386b2b493f8fc9f1f0660004f296dd1a511e3ae8a" Sep 09 07:07:35.493601 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.493579 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-4z6z4"] Sep 09 07:07:35.498546 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.498530 2559 scope.go:117] "RemoveContainer" containerID="c011de24d22196e9de5cf4d803b4ddbc6a03e064ae63b2542a54ae2f5132a1ac" Sep 09 07:07:35.505799 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.505782 2559 scope.go:117] "RemoveContainer" containerID="125225ed79b941ceb8926796013778ac4ca16802594f2ee95d968a94ee4fc479" Sep 09 07:07:35.506086 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:07:35.506067 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"125225ed79b941ceb8926796013778ac4ca16802594f2ee95d968a94ee4fc479\": container with ID starting with 125225ed79b941ceb8926796013778ac4ca16802594f2ee95d968a94ee4fc479 not found: ID does not exist" containerID="125225ed79b941ceb8926796013778ac4ca16802594f2ee95d968a94ee4fc479" Sep 09 07:07:35.506170 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.506094 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"125225ed79b941ceb8926796013778ac4ca16802594f2ee95d968a94ee4fc479"} err="failed to get container status \"125225ed79b941ceb8926796013778ac4ca16802594f2ee95d968a94ee4fc479\": rpc error: code = NotFound desc = could not find container \"125225ed79b941ceb8926796013778ac4ca16802594f2ee95d968a94ee4fc479\": container with ID starting with 125225ed79b941ceb8926796013778ac4ca16802594f2ee95d968a94ee4fc479 not found: ID does not exist" Sep 09 07:07:35.506170 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.506110 2559 scope.go:117] "RemoveContainer" containerID="b67c731c0e4ac190f69200fd5c2640445541d9fb4131a1912935ac82382f12ef" Sep 09 07:07:35.506364 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:07:35.506343 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b67c731c0e4ac190f69200fd5c2640445541d9fb4131a1912935ac82382f12ef\": container with ID starting with b67c731c0e4ac190f69200fd5c2640445541d9fb4131a1912935ac82382f12ef not found: ID does not exist" containerID="b67c731c0e4ac190f69200fd5c2640445541d9fb4131a1912935ac82382f12ef" Sep 09 07:07:35.506408 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.506369 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b67c731c0e4ac190f69200fd5c2640445541d9fb4131a1912935ac82382f12ef"} err="failed to get container status \"b67c731c0e4ac190f69200fd5c2640445541d9fb4131a1912935ac82382f12ef\": rpc error: code = NotFound desc = could not find container \"b67c731c0e4ac190f69200fd5c2640445541d9fb4131a1912935ac82382f12ef\": container with ID starting with b67c731c0e4ac190f69200fd5c2640445541d9fb4131a1912935ac82382f12ef not found: ID does not exist" Sep 09 07:07:35.506408 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.506385 2559 scope.go:117] "RemoveContainer" containerID="1cae1ad2e337fde2d70d17eba944bee8a03ec616b938356c8ad4317d1b196df4" Sep 09 07:07:35.506634 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:07:35.506617 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cae1ad2e337fde2d70d17eba944bee8a03ec616b938356c8ad4317d1b196df4\": container with ID starting with 1cae1ad2e337fde2d70d17eba944bee8a03ec616b938356c8ad4317d1b196df4 not found: ID does not exist" containerID="1cae1ad2e337fde2d70d17eba944bee8a03ec616b938356c8ad4317d1b196df4" Sep 09 07:07:35.506687 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.506638 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cae1ad2e337fde2d70d17eba944bee8a03ec616b938356c8ad4317d1b196df4"} err="failed to get container status \"1cae1ad2e337fde2d70d17eba944bee8a03ec616b938356c8ad4317d1b196df4\": rpc error: code = NotFound desc = could not find container \"1cae1ad2e337fde2d70d17eba944bee8a03ec616b938356c8ad4317d1b196df4\": container with ID starting with 1cae1ad2e337fde2d70d17eba944bee8a03ec616b938356c8ad4317d1b196df4 not found: ID does not exist" Sep 09 07:07:35.506687 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.506657 2559 scope.go:117] "RemoveContainer" containerID="6a6a9eab55b65eea2eda481386b2b493f8fc9f1f0660004f296dd1a511e3ae8a" Sep 09 07:07:35.506883 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:07:35.506867 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a6a9eab55b65eea2eda481386b2b493f8fc9f1f0660004f296dd1a511e3ae8a\": container with ID starting with 6a6a9eab55b65eea2eda481386b2b493f8fc9f1f0660004f296dd1a511e3ae8a not found: ID does not exist" containerID="6a6a9eab55b65eea2eda481386b2b493f8fc9f1f0660004f296dd1a511e3ae8a" Sep 09 07:07:35.506926 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.506888 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a6a9eab55b65eea2eda481386b2b493f8fc9f1f0660004f296dd1a511e3ae8a"} err="failed to get container status \"6a6a9eab55b65eea2eda481386b2b493f8fc9f1f0660004f296dd1a511e3ae8a\": rpc error: code = NotFound desc = could not find container \"6a6a9eab55b65eea2eda481386b2b493f8fc9f1f0660004f296dd1a511e3ae8a\": container with ID starting with 6a6a9eab55b65eea2eda481386b2b493f8fc9f1f0660004f296dd1a511e3ae8a not found: ID does not exist" Sep 09 07:07:35.506926 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.506900 2559 scope.go:117] "RemoveContainer" containerID="c011de24d22196e9de5cf4d803b4ddbc6a03e064ae63b2542a54ae2f5132a1ac" Sep 09 07:07:35.507163 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:07:35.507144 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c011de24d22196e9de5cf4d803b4ddbc6a03e064ae63b2542a54ae2f5132a1ac\": container with ID starting with c011de24d22196e9de5cf4d803b4ddbc6a03e064ae63b2542a54ae2f5132a1ac not found: ID does not exist" containerID="c011de24d22196e9de5cf4d803b4ddbc6a03e064ae63b2542a54ae2f5132a1ac" Sep 09 07:07:35.507225 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.507167 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c011de24d22196e9de5cf4d803b4ddbc6a03e064ae63b2542a54ae2f5132a1ac"} err="failed to get container status \"c011de24d22196e9de5cf4d803b4ddbc6a03e064ae63b2542a54ae2f5132a1ac\": rpc error: code = NotFound desc = could not find container \"c011de24d22196e9de5cf4d803b4ddbc6a03e064ae63b2542a54ae2f5132a1ac\": container with ID starting with c011de24d22196e9de5cf4d803b4ddbc6a03e064ae63b2542a54ae2f5132a1ac not found: ID does not exist" Sep 09 07:07:35.626571 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.626500 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d6rdz\" (UniqueName: \"kubernetes.io/projected/b4f53b83-9137-4998-b3b2-bea46e694595-kube-api-access-d6rdz\") pod \"b4f53b83-9137-4998-b3b2-bea46e694595\" (UID: \"b4f53b83-9137-4998-b3b2-bea46e694595\") " Sep 09 07:07:35.626571 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.626538 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b4f53b83-9137-4998-b3b2-bea46e694595-feast-data\") pod \"b4f53b83-9137-4998-b3b2-bea46e694595\" (UID: \"b4f53b83-9137-4998-b3b2-bea46e694595\") " Sep 09 07:07:35.626777 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.626582 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b4f53b83-9137-4998-b3b2-bea46e694595-online-tls\") pod \"b4f53b83-9137-4998-b3b2-bea46e694595\" (UID: \"b4f53b83-9137-4998-b3b2-bea46e694595\") " Sep 09 07:07:35.626777 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.626604 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b4f53b83-9137-4998-b3b2-bea46e694595-ui-tls\") pod \"b4f53b83-9137-4998-b3b2-bea46e694595\" (UID: \"b4f53b83-9137-4998-b3b2-bea46e694595\") " Sep 09 07:07:35.626777 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.626621 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b4f53b83-9137-4998-b3b2-bea46e694595-offline-tls\") pod \"b4f53b83-9137-4998-b3b2-bea46e694595\" (UID: \"b4f53b83-9137-4998-b3b2-bea46e694595\") " Sep 09 07:07:35.626777 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.626659 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b4f53b83-9137-4998-b3b2-bea46e694595-registry-tls\") pod \"b4f53b83-9137-4998-b3b2-bea46e694595\" (UID: \"b4f53b83-9137-4998-b3b2-bea46e694595\") " Sep 09 07:07:35.627233 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.627204 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4f53b83-9137-4998-b3b2-bea46e694595-feast-data" (OuterVolumeSpecName: "feast-data") pod "b4f53b83-9137-4998-b3b2-bea46e694595" (UID: "b4f53b83-9137-4998-b3b2-bea46e694595"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 07:07:35.628748 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.628723 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4f53b83-9137-4998-b3b2-bea46e694595-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "b4f53b83-9137-4998-b3b2-bea46e694595" (UID: "b4f53b83-9137-4998-b3b2-bea46e694595"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 07:07:35.628748 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.628742 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4f53b83-9137-4998-b3b2-bea46e694595-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "b4f53b83-9137-4998-b3b2-bea46e694595" (UID: "b4f53b83-9137-4998-b3b2-bea46e694595"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 07:07:35.628923 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.628727 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4f53b83-9137-4998-b3b2-bea46e694595-kube-api-access-d6rdz" (OuterVolumeSpecName: "kube-api-access-d6rdz") pod "b4f53b83-9137-4998-b3b2-bea46e694595" (UID: "b4f53b83-9137-4998-b3b2-bea46e694595"). InnerVolumeSpecName "kube-api-access-d6rdz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 07:07:35.629086 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.629069 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4f53b83-9137-4998-b3b2-bea46e694595-online-tls" (OuterVolumeSpecName: "online-tls") pod "b4f53b83-9137-4998-b3b2-bea46e694595" (UID: "b4f53b83-9137-4998-b3b2-bea46e694595"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 07:07:35.629141 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.629096 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4f53b83-9137-4998-b3b2-bea46e694595-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "b4f53b83-9137-4998-b3b2-bea46e694595" (UID: "b4f53b83-9137-4998-b3b2-bea46e694595"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 07:07:35.728123 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.728091 2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b4f53b83-9137-4998-b3b2-bea46e694595-registry-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:07:35.728123 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.728123 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6rdz\" (UniqueName: \"kubernetes.io/projected/b4f53b83-9137-4998-b3b2-bea46e694595-kube-api-access-d6rdz\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:07:35.728123 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.728132 2559 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b4f53b83-9137-4998-b3b2-bea46e694595-feast-data\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:07:35.728277 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.728142 2559 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b4f53b83-9137-4998-b3b2-bea46e694595-online-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:07:35.728277 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.728150 2559 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b4f53b83-9137-4998-b3b2-bea46e694595-ui-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:07:35.728277 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:35.728157 2559 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b4f53b83-9137-4998-b3b2-bea46e694595-offline-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:07:36.476150 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:36.476118 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" event={"ID":"b4f53b83-9137-4998-b3b2-bea46e694595","Type":"ContainerDied","Data":"553f687b187f6ff0f93d5db51851c4ae974af33d9ede0c62fe2d815a323054b9"} Sep 09 07:07:36.476582 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:36.476168 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx" Sep 09 07:07:36.476582 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:36.476176 2559 scope.go:117] "RemoveContainer" containerID="16df504799e061bdbce19d9f30879c994b26ee7198bac76dd0d88c4e057d9897" Sep 09 07:07:36.484702 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:36.484683 2559 scope.go:117] "RemoveContainer" containerID="98d4b97907afa776bdd9d6ae7ea642008a953f94f1b4631192d1fcb783fb30f2" Sep 09 07:07:36.492161 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:36.492145 2559 scope.go:117] "RemoveContainer" containerID="6bc5cd7d29647903ebbb1d24b4c50aaa93b3c8d6c224b93b75d5310d256fcd0c" Sep 09 07:07:36.498142 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:36.498120 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx"] Sep 09 07:07:36.499753 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:36.499732 2559 scope.go:117] "RemoveContainer" containerID="e2568c0847b7d49e0b4d754bce7d196caff0ff03feaa4684878a1c67e8659469" Sep 09 07:07:36.501940 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:36.501921 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-q5fpx"] Sep 09 07:07:36.507211 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:36.507193 2559 scope.go:117] "RemoveContainer" containerID="407ad025e050c34b5c57c5ca183939646cd6412a9d7f135d8a92afd9b9075018" Sep 09 07:07:36.514100 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:36.514080 2559 scope.go:117] "RemoveContainer" containerID="6b9812a8a055e2ea219e16b6989eb6ebe9dcc64c4e3ce9262e52f568757444bc" Sep 09 07:07:36.701670 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:36.701632 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72aca48e-fd59-4130-b4f5-570a5e0629d3" path="/var/lib/kubelet/pods/72aca48e-fd59-4130-b4f5-570a5e0629d3/volumes" Sep 09 07:07:36.702199 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:07:36.702185 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4f53b83-9137-4998-b3b2-bea46e694595" path="/var/lib/kubelet/pods/b4f53b83-9137-4998-b3b2-bea46e694595/volumes" Sep 09 07:08:16.064182 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.064096 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg"] Sep 09 07:08:16.064637 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.064588 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4f53b83-9137-4998-b3b2-bea46e694595" containerName="offline" Sep 09 07:08:16.064637 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.064607 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4f53b83-9137-4998-b3b2-bea46e694595" containerName="offline" Sep 09 07:08:16.064637 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.064627 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4f53b83-9137-4998-b3b2-bea46e694595" containerName="online" Sep 09 07:08:16.064637 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.064636 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4f53b83-9137-4998-b3b2-bea46e694595" containerName="online" Sep 09 07:08:16.064782 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.064648 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4f53b83-9137-4998-b3b2-bea46e694595" containerName="feast-init" Sep 09 07:08:16.064782 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.064657 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4f53b83-9137-4998-b3b2-bea46e694595" containerName="feast-init" Sep 09 07:08:16.064782 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.064672 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4f53b83-9137-4998-b3b2-bea46e694595" containerName="ui" Sep 09 07:08:16.064782 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.064680 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4f53b83-9137-4998-b3b2-bea46e694595" containerName="ui" Sep 09 07:08:16.064782 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.064694 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72aca48e-fd59-4130-b4f5-570a5e0629d3" containerName="ui" Sep 09 07:08:16.064782 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.064703 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="72aca48e-fd59-4130-b4f5-570a5e0629d3" containerName="ui" Sep 09 07:08:16.064782 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.064717 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4f53b83-9137-4998-b3b2-bea46e694595" containerName="feast-apply" Sep 09 07:08:16.064782 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.064725 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4f53b83-9137-4998-b3b2-bea46e694595" containerName="feast-apply" Sep 09 07:08:16.064782 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.064735 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72aca48e-fd59-4130-b4f5-570a5e0629d3" containerName="feast-apply" Sep 09 07:08:16.064782 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.064743 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="72aca48e-fd59-4130-b4f5-570a5e0629d3" containerName="feast-apply" Sep 09 07:08:16.064782 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.064759 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72aca48e-fd59-4130-b4f5-570a5e0629d3" containerName="feast-init" Sep 09 07:08:16.064782 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.064768 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="72aca48e-fd59-4130-b4f5-570a5e0629d3" containerName="feast-init" Sep 09 07:08:16.064782 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.064779 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72aca48e-fd59-4130-b4f5-570a5e0629d3" containerName="online" Sep 09 07:08:16.065206 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.064789 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="72aca48e-fd59-4130-b4f5-570a5e0629d3" containerName="online" Sep 09 07:08:16.065206 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.064801 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72aca48e-fd59-4130-b4f5-570a5e0629d3" containerName="offline" Sep 09 07:08:16.065206 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.064810 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="72aca48e-fd59-4130-b4f5-570a5e0629d3" containerName="offline" Sep 09 07:08:16.065206 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.064828 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4f53b83-9137-4998-b3b2-bea46e694595" containerName="registry" Sep 09 07:08:16.065206 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.064837 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4f53b83-9137-4998-b3b2-bea46e694595" containerName="registry" Sep 09 07:08:16.065206 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.064912 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4f53b83-9137-4998-b3b2-bea46e694595" containerName="registry" Sep 09 07:08:16.065206 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.064922 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4f53b83-9137-4998-b3b2-bea46e694595" containerName="offline" Sep 09 07:08:16.065206 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.064932 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4f53b83-9137-4998-b3b2-bea46e694595" containerName="ui" Sep 09 07:08:16.065206 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.064946 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4f53b83-9137-4998-b3b2-bea46e694595" containerName="online" Sep 09 07:08:16.065206 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.064956 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="72aca48e-fd59-4130-b4f5-570a5e0629d3" containerName="offline" Sep 09 07:08:16.065206 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.064968 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="72aca48e-fd59-4130-b4f5-570a5e0629d3" containerName="online" Sep 09 07:08:16.065206 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.064977 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="72aca48e-fd59-4130-b4f5-570a5e0629d3" containerName="ui" Sep 09 07:08:16.070367 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.070342 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:08:16.073476 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.073455 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 09 07:08:16.073681 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.073642 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 09 07:08:16.073681 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.073648 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-phb4r\"" Sep 09 07:08:16.073898 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.073793 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 09 07:08:16.074043 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.074022 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 09 07:08:16.074447 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.074423 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 09 07:08:16.074628 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.074612 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 09 07:08:16.082099 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.082078 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg"] Sep 09 07:08:16.166176 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.166141 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/213f3de8-84fc-4f01-9c00-59d846385367-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xqckg\" (UID: \"213f3de8-84fc-4f01-9c00-59d846385367\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:08:16.166176 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.166175 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdp69\" (UniqueName: \"kubernetes.io/projected/213f3de8-84fc-4f01-9c00-59d846385367-kube-api-access-fdp69\") pod \"feast-simple-feast-setup-57576b8fcf-xqckg\" (UID: \"213f3de8-84fc-4f01-9c00-59d846385367\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:08:16.166415 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.166202 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/213f3de8-84fc-4f01-9c00-59d846385367-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xqckg\" (UID: \"213f3de8-84fc-4f01-9c00-59d846385367\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:08:16.166415 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.166226 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/213f3de8-84fc-4f01-9c00-59d846385367-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xqckg\" (UID: \"213f3de8-84fc-4f01-9c00-59d846385367\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:08:16.166415 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.166295 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/213f3de8-84fc-4f01-9c00-59d846385367-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xqckg\" (UID: \"213f3de8-84fc-4f01-9c00-59d846385367\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:08:16.166415 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.166342 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/213f3de8-84fc-4f01-9c00-59d846385367-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-xqckg\" (UID: \"213f3de8-84fc-4f01-9c00-59d846385367\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:08:16.267776 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.267740 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/213f3de8-84fc-4f01-9c00-59d846385367-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xqckg\" (UID: \"213f3de8-84fc-4f01-9c00-59d846385367\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:08:16.267776 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.267780 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdp69\" (UniqueName: \"kubernetes.io/projected/213f3de8-84fc-4f01-9c00-59d846385367-kube-api-access-fdp69\") pod \"feast-simple-feast-setup-57576b8fcf-xqckg\" (UID: \"213f3de8-84fc-4f01-9c00-59d846385367\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:08:16.268079 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.267815 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/213f3de8-84fc-4f01-9c00-59d846385367-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xqckg\" (UID: \"213f3de8-84fc-4f01-9c00-59d846385367\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:08:16.268079 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.267846 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/213f3de8-84fc-4f01-9c00-59d846385367-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xqckg\" (UID: \"213f3de8-84fc-4f01-9c00-59d846385367\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:08:16.268079 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:08:16.267917 2559 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Sep 09 07:08:16.268079 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:08:16.268002 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/213f3de8-84fc-4f01-9c00-59d846385367-offline-tls podName:213f3de8-84fc-4f01-9c00-59d846385367 nodeName:}" failed. No retries permitted until 2026-09-09 07:08:16.767977479 +0000 UTC m=+1794.664394879 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/213f3de8-84fc-4f01-9c00-59d846385367-offline-tls") pod "feast-simple-feast-setup-57576b8fcf-xqckg" (UID: "213f3de8-84fc-4f01-9c00-59d846385367") : secret "feast-simple-feast-setup-offline-tls" not found Sep 09 07:08:16.268079 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.268027 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/213f3de8-84fc-4f01-9c00-59d846385367-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xqckg\" (UID: \"213f3de8-84fc-4f01-9c00-59d846385367\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:08:16.268079 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.268078 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/213f3de8-84fc-4f01-9c00-59d846385367-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-xqckg\" (UID: \"213f3de8-84fc-4f01-9c00-59d846385367\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:08:16.268484 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.268464 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/213f3de8-84fc-4f01-9c00-59d846385367-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-xqckg\" (UID: \"213f3de8-84fc-4f01-9c00-59d846385367\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:08:16.270423 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.270399 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/213f3de8-84fc-4f01-9c00-59d846385367-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xqckg\" (UID: \"213f3de8-84fc-4f01-9c00-59d846385367\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:08:16.270423 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.270402 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/213f3de8-84fc-4f01-9c00-59d846385367-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xqckg\" (UID: \"213f3de8-84fc-4f01-9c00-59d846385367\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:08:16.270581 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.270497 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/213f3de8-84fc-4f01-9c00-59d846385367-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xqckg\" (UID: \"213f3de8-84fc-4f01-9c00-59d846385367\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:08:16.277260 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.277235 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdp69\" (UniqueName: \"kubernetes.io/projected/213f3de8-84fc-4f01-9c00-59d846385367-kube-api-access-fdp69\") pod \"feast-simple-feast-setup-57576b8fcf-xqckg\" (UID: \"213f3de8-84fc-4f01-9c00-59d846385367\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:08:16.773839 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:16.773798 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/213f3de8-84fc-4f01-9c00-59d846385367-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xqckg\" (UID: \"213f3de8-84fc-4f01-9c00-59d846385367\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:08:16.774023 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:08:16.773936 2559 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Sep 09 07:08:16.774023 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:08:16.774015 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/213f3de8-84fc-4f01-9c00-59d846385367-offline-tls podName:213f3de8-84fc-4f01-9c00-59d846385367 nodeName:}" failed. No retries permitted until 2026-09-09 07:08:17.773997704 +0000 UTC m=+1795.670415109 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/213f3de8-84fc-4f01-9c00-59d846385367-offline-tls") pod "feast-simple-feast-setup-57576b8fcf-xqckg" (UID: "213f3de8-84fc-4f01-9c00-59d846385367") : secret "feast-simple-feast-setup-offline-tls" not found Sep 09 07:08:17.783637 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:17.783599 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/213f3de8-84fc-4f01-9c00-59d846385367-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xqckg\" (UID: \"213f3de8-84fc-4f01-9c00-59d846385367\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:08:17.786293 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:17.786266 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/213f3de8-84fc-4f01-9c00-59d846385367-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xqckg\" (UID: \"213f3de8-84fc-4f01-9c00-59d846385367\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:08:17.892067 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:17.892029 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:08:18.170080 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:18.170040 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg"] Sep 09 07:08:18.172676 ip-10-0-132-137 kubenswrapper[2559]: W0909 07:08:18.172644 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod213f3de8_84fc_4f01_9c00_59d846385367.slice/crio-72ac7826471296e1e01310ece0ba229fd31140f4e2b041147ed72759dc5374ee WatchSource:0}: Error finding container 72ac7826471296e1e01310ece0ba229fd31140f4e2b041147ed72759dc5374ee: Status 404 returned error can't find the container with id 72ac7826471296e1e01310ece0ba229fd31140f4e2b041147ed72759dc5374ee Sep 09 07:08:18.648107 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:18.648069 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" event={"ID":"213f3de8-84fc-4f01-9c00-59d846385367","Type":"ContainerStarted","Data":"e6b935efc4b0d00fa630d7bc95cf6b421d1d3761cceaf296ffd865c086baf7c6"} Sep 09 07:08:18.648288 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:18.648119 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" event={"ID":"213f3de8-84fc-4f01-9c00-59d846385367","Type":"ContainerStarted","Data":"72ac7826471296e1e01310ece0ba229fd31140f4e2b041147ed72759dc5374ee"} Sep 09 07:08:22.617525 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:22.617490 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nf9gf_26d28788-4f98-454b-ab39-2d0d857aa674/console-operator/2.log" Sep 09 07:08:22.622235 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:22.622213 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nf9gf_26d28788-4f98-454b-ab39-2d0d857aa674/console-operator/2.log" Sep 09 07:08:23.675242 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:23.675208 2559 generic.go:356] "Generic (PLEG): container finished" podID="213f3de8-84fc-4f01-9c00-59d846385367" containerID="e6b935efc4b0d00fa630d7bc95cf6b421d1d3761cceaf296ffd865c086baf7c6" exitCode=0 Sep 09 07:08:23.675614 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:23.675264 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" event={"ID":"213f3de8-84fc-4f01-9c00-59d846385367","Type":"ContainerDied","Data":"e6b935efc4b0d00fa630d7bc95cf6b421d1d3761cceaf296ffd865c086baf7c6"} Sep 09 07:08:24.683444 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:24.683411 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" event={"ID":"213f3de8-84fc-4f01-9c00-59d846385367","Type":"ContainerStarted","Data":"351fad40a5d68dac9fdf08fb06deedd8995c1ac6740c86f8777b147a71a23410"} Sep 09 07:08:29.706681 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:29.706648 2559 generic.go:356] "Generic (PLEG): container finished" podID="213f3de8-84fc-4f01-9c00-59d846385367" containerID="351fad40a5d68dac9fdf08fb06deedd8995c1ac6740c86f8777b147a71a23410" exitCode=0 Sep 09 07:08:29.707191 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:29.706728 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" event={"ID":"213f3de8-84fc-4f01-9c00-59d846385367","Type":"ContainerDied","Data":"351fad40a5d68dac9fdf08fb06deedd8995c1ac6740c86f8777b147a71a23410"} Sep 09 07:08:30.716641 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:30.716590 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" event={"ID":"213f3de8-84fc-4f01-9c00-59d846385367","Type":"ContainerStarted","Data":"2d3fd844e10832badac1884e9210b90aec3f4ec0de60ee2b5182a7ae9d12d557"} Sep 09 07:08:30.717159 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:30.716659 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" event={"ID":"213f3de8-84fc-4f01-9c00-59d846385367","Type":"ContainerStarted","Data":"f4813ffd6e5be421709a7411348ee8f6e8066bb28b8329fbcb86bccca59fb520"} Sep 09 07:08:30.717159 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:30.716683 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" event={"ID":"213f3de8-84fc-4f01-9c00-59d846385367","Type":"ContainerStarted","Data":"edaaa448d1596eec9a0675b548af6f478ba2feecc8a20ec36c9dc1fae32eb673"} Sep 09 07:08:30.717159 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:30.716703 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" event={"ID":"213f3de8-84fc-4f01-9c00-59d846385367","Type":"ContainerStarted","Data":"a9cacdfd26ce80f2067b91e17fe52c904c83d7a6be218b3cc9a3fbd65e594fa6"} Sep 09 07:08:30.740978 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:30.740916 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" podStartSLOduration=14.740892701 podStartE2EDuration="14.740892701s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:08:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:08:30.738394191 +0000 UTC m=+1808.634811628" watchObservedRunningTime="2026-09-09 07:08:30.740892701 +0000 UTC m=+1808.637310124" Sep 09 07:08:32.892572 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:32.892470 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:08:32.892572 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:32.892533 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:08:32.892572 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:32.892554 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:08:32.892572 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:32.892570 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:08:32.894637 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:32.894604 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" podUID="213f3de8-84fc-4f01-9c00-59d846385367" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.43:6571: connect: connection refused" Sep 09 07:08:32.894871 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:32.894639 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" podUID="213f3de8-84fc-4f01-9c00-59d846385367" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.43:8816: connect: connection refused" Sep 09 07:08:32.894998 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:32.894637 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" podUID="213f3de8-84fc-4f01-9c00-59d846385367" containerName="online" probeResult="failure" output="Get \"https://10.132.0.43:6567/health\": dial tcp 10.132.0.43:6567: connect: connection refused" Sep 09 07:08:32.894998 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:32.894652 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" podUID="213f3de8-84fc-4f01-9c00-59d846385367" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.43:8443: connect: connection refused" Sep 09 07:08:35.893028 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:35.892958 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" podUID="213f3de8-84fc-4f01-9c00-59d846385367" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.43:6571: connect: connection refused" Sep 09 07:08:35.893028 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:35.893006 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" podUID="213f3de8-84fc-4f01-9c00-59d846385367" containerName="online" probeResult="failure" output="Get \"https://10.132.0.43:6567/health\": dial tcp 10.132.0.43:6567: connect: connection refused" Sep 09 07:08:35.893593 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:35.892975 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" podUID="213f3de8-84fc-4f01-9c00-59d846385367" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.43:8816: connect: connection refused" Sep 09 07:08:35.893593 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:35.892975 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" podUID="213f3de8-84fc-4f01-9c00-59d846385367" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.43:8443: connect: connection refused" Sep 09 07:08:37.893192 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:37.893156 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:08:37.893584 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:37.893224 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:08:37.893584 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:37.893245 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:08:37.893584 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:37.893268 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:08:38.893405 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:38.893374 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:08:38.893814 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:38.893598 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:08:38.893814 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:38.893788 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:08:38.894374 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:38.894349 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:08:38.894521 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:38.894406 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:08:38.894693 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:38.894675 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:08:38.898941 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:38.898920 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:08:39.770328 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:08:39.770292 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:09:41.897739 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:41.897705 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg"] Sep 09 07:09:41.898334 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:41.898006 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" podUID="213f3de8-84fc-4f01-9c00-59d846385367" containerName="registry" containerID="cri-o://a9cacdfd26ce80f2067b91e17fe52c904c83d7a6be218b3cc9a3fbd65e594fa6" gracePeriod=30 Sep 09 07:09:41.898334 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:41.898122 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" podUID="213f3de8-84fc-4f01-9c00-59d846385367" containerName="ui" containerID="cri-o://2d3fd844e10832badac1884e9210b90aec3f4ec0de60ee2b5182a7ae9d12d557" gracePeriod=30 Sep 09 07:09:41.898334 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:41.898156 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" podUID="213f3de8-84fc-4f01-9c00-59d846385367" containerName="online" containerID="cri-o://edaaa448d1596eec9a0675b548af6f478ba2feecc8a20ec36c9dc1fae32eb673" gracePeriod=30 Sep 09 07:09:41.898334 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:41.898127 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" podUID="213f3de8-84fc-4f01-9c00-59d846385367" containerName="offline" containerID="cri-o://f4813ffd6e5be421709a7411348ee8f6e8066bb28b8329fbcb86bccca59fb520" gracePeriod=30 Sep 09 07:09:42.309886 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:42.309846 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l"] Sep 09 07:09:42.313700 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:42.313680 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:09:42.316125 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:42.316099 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-h85nh\"" Sep 09 07:09:42.325035 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:42.325012 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l"] Sep 09 07:09:42.479819 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:42.479783 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-gx48l\" (UID: \"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:09:42.479940 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:42.479832 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-gx48l\" (UID: \"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:09:42.479940 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:42.479893 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpdtp\" (UniqueName: \"kubernetes.io/projected/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-kube-api-access-jpdtp\") pod \"feast-simple-feast-setup-57576b8fcf-gx48l\" (UID: \"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:09:42.480016 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:42.479988 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-gx48l\" (UID: \"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:09:42.480016 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:42.480007 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-gx48l\" (UID: \"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:09:42.480118 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:42.480049 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-gx48l\" (UID: \"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:09:42.581171 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:42.581078 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-gx48l\" (UID: \"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:09:42.581171 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:42.581138 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-gx48l\" (UID: \"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:09:42.581171 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:42.581166 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpdtp\" (UniqueName: \"kubernetes.io/projected/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-kube-api-access-jpdtp\") pod \"feast-simple-feast-setup-57576b8fcf-gx48l\" (UID: \"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:09:42.581476 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:09:42.581205 2559 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found Sep 09 07:09:42.581476 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:42.581243 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-gx48l\" (UID: \"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:09:42.581476 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:42.581266 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-gx48l\" (UID: \"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:09:42.581476 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:09:42.581290 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-online-tls podName:4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d nodeName:}" failed. No retries permitted until 2026-09-09 07:09:43.081269428 +0000 UTC m=+1880.977686832 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-online-tls") pod "feast-simple-feast-setup-57576b8fcf-gx48l" (UID: "4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d") : secret "feast-simple-feast-setup-online-tls" not found Sep 09 07:09:42.581476 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:42.581327 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-gx48l\" (UID: \"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:09:42.581476 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:09:42.581355 2559 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Sep 09 07:09:42.581476 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:09:42.581419 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-registry-tls podName:4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d nodeName:}" failed. No retries permitted until 2026-09-09 07:09:43.081397984 +0000 UTC m=+1880.977815398 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-registry-tls") pod "feast-simple-feast-setup-57576b8fcf-gx48l" (UID: "4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d") : secret "feast-simple-feast-setup-registry-tls" not found Sep 09 07:09:42.581801 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:42.581685 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-gx48l\" (UID: \"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:09:42.583578 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:42.583553 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-gx48l\" (UID: \"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:09:42.583729 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:42.583709 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-gx48l\" (UID: \"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:09:42.591381 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:42.591355 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpdtp\" (UniqueName: \"kubernetes.io/projected/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-kube-api-access-jpdtp\") pod \"feast-simple-feast-setup-57576b8fcf-gx48l\" (UID: \"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:09:43.032073 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:43.032022 2559 generic.go:356] "Generic (PLEG): container finished" podID="213f3de8-84fc-4f01-9c00-59d846385367" containerID="2d3fd844e10832badac1884e9210b90aec3f4ec0de60ee2b5182a7ae9d12d557" exitCode=0 Sep 09 07:09:43.032073 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:43.032047 2559 generic.go:356] "Generic (PLEG): container finished" podID="213f3de8-84fc-4f01-9c00-59d846385367" containerID="edaaa448d1596eec9a0675b548af6f478ba2feecc8a20ec36c9dc1fae32eb673" exitCode=0 Sep 09 07:09:43.032484 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:43.032095 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" event={"ID":"213f3de8-84fc-4f01-9c00-59d846385367","Type":"ContainerDied","Data":"2d3fd844e10832badac1884e9210b90aec3f4ec0de60ee2b5182a7ae9d12d557"} Sep 09 07:09:43.032484 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:43.032140 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" event={"ID":"213f3de8-84fc-4f01-9c00-59d846385367","Type":"ContainerDied","Data":"edaaa448d1596eec9a0675b548af6f478ba2feecc8a20ec36c9dc1fae32eb673"} Sep 09 07:09:43.085444 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:43.085410 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-gx48l\" (UID: \"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:09:43.085594 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:43.085479 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-gx48l\" (UID: \"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:09:43.087904 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:43.087872 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-gx48l\" (UID: \"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:09:43.087904 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:43.087893 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-gx48l\" (UID: \"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:09:43.223037 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:43.222999 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:09:43.499698 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:43.499672 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l"] Sep 09 07:09:43.500918 ip-10-0-132-137 kubenswrapper[2559]: W0909 07:09:43.500888 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e9a3ccc_ad34_49d7_9578_7eaab7ef9c7d.slice/crio-7334ccd74609596cca0c8eaba8e62dfb0d988479ad0125fbf76680faa4859768 WatchSource:0}: Error finding container 7334ccd74609596cca0c8eaba8e62dfb0d988479ad0125fbf76680faa4859768: Status 404 returned error can't find the container with id 7334ccd74609596cca0c8eaba8e62dfb0d988479ad0125fbf76680faa4859768 Sep 09 07:09:44.037924 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:44.037880 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" event={"ID":"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d","Type":"ContainerStarted","Data":"d7d36f04cb963185c96da0a0ba67f9390847bfc5891d959b90c003cbd81c7c04"} Sep 09 07:09:44.038368 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:44.037938 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" event={"ID":"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d","Type":"ContainerStarted","Data":"7334ccd74609596cca0c8eaba8e62dfb0d988479ad0125fbf76680faa4859768"} Sep 09 07:09:48.894820 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:48.894770 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" podUID="213f3de8-84fc-4f01-9c00-59d846385367" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.43:8443: connect: connection refused" Sep 09 07:09:49.063660 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:49.063624 2559 generic.go:356] "Generic (PLEG): container finished" podID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerID="d7d36f04cb963185c96da0a0ba67f9390847bfc5891d959b90c003cbd81c7c04" exitCode=0 Sep 09 07:09:49.063841 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:49.063685 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" event={"ID":"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d","Type":"ContainerDied","Data":"d7d36f04cb963185c96da0a0ba67f9390847bfc5891d959b90c003cbd81c7c04"} Sep 09 07:09:49.767273 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:49.767229 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" podUID="213f3de8-84fc-4f01-9c00-59d846385367" containerName="online" probeResult="failure" output="Get \"https://10.132.0.43:6567/health\": dial tcp 10.132.0.43:6567: connect: connection refused" Sep 09 07:09:50.076439 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:50.076337 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" event={"ID":"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d","Type":"ContainerStarted","Data":"d622af38b96eac078bcb5ed370ec300c16f58815cdf3732b58ffb2ae24b1e782"} Sep 09 07:09:55.101744 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:55.101653 2559 generic.go:356] "Generic (PLEG): container finished" podID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerID="d622af38b96eac078bcb5ed370ec300c16f58815cdf3732b58ffb2ae24b1e782" exitCode=0 Sep 09 07:09:55.102242 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:55.101730 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" event={"ID":"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d","Type":"ContainerDied","Data":"d622af38b96eac078bcb5ed370ec300c16f58815cdf3732b58ffb2ae24b1e782"} Sep 09 07:09:56.113092 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:56.113031 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" event={"ID":"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d","Type":"ContainerStarted","Data":"b9a0a997cd490f85574a6e32651216150b77bccc448cbd239eb306e79b78882e"} Sep 09 07:09:56.113585 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:56.113116 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" event={"ID":"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d","Type":"ContainerStarted","Data":"f649e8fe7419529acbd847a89d949beddec0b031599268d9b7ad78f52f3441d3"} Sep 09 07:09:56.113585 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:56.113134 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" event={"ID":"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d","Type":"ContainerStarted","Data":"c0e8043ac18aa9448ebbe67c770fa31f9b4a05661d8d9e482e93ed59fa8351cf"} Sep 09 07:09:56.113585 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:56.113152 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" event={"ID":"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d","Type":"ContainerStarted","Data":"b03b78d6584000389b7a04a1a96fdc547fb0f8468d3bbec70346d1d63c1fba24"} Sep 09 07:09:56.139945 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:56.139877 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" podStartSLOduration=14.139853363 podStartE2EDuration="14.139853363s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:09:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:09:56.138283072 +0000 UTC m=+1894.034700506" watchObservedRunningTime="2026-09-09 07:09:56.139853363 +0000 UTC m=+1894.036270786" Sep 09 07:09:58.223464 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:58.223391 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:09:58.223972 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:58.223475 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:09:58.223972 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:58.223496 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:09:58.223972 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:58.223513 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:09:58.225582 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:58.225474 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" podUID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.44:8443: connect: connection refused" Sep 09 07:09:58.225582 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:58.225504 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" podUID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.44:8816: connect: connection refused" Sep 09 07:09:58.225582 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:58.225519 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" podUID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerName="online" probeResult="failure" output="Get \"https://10.132.0.44:6567/health\": dial tcp 10.132.0.44:6567: connect: connection refused" Sep 09 07:09:58.225582 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:58.225527 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" podUID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.44:6571: connect: connection refused" Sep 09 07:09:58.895599 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:58.895175 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" podUID="213f3de8-84fc-4f01-9c00-59d846385367" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.43:8443: connect: connection refused" Sep 09 07:09:59.767848 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:09:59.767791 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" podUID="213f3de8-84fc-4f01-9c00-59d846385367" containerName="online" probeResult="failure" output="Get \"https://10.132.0.43:6567/health\": dial tcp 10.132.0.43:6567: connect: connection refused" Sep 09 07:10:01.224299 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:01.224252 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" podUID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.44:8443: connect: connection refused" Sep 09 07:10:01.224932 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:01.224264 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" podUID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerName="online" probeResult="failure" output="Get \"https://10.132.0.44:6567/health\": dial tcp 10.132.0.44:6567: connect: connection refused" Sep 09 07:10:01.225094 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:01.224258 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" podUID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.44:6571: connect: connection refused" Sep 09 07:10:01.225094 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:01.224332 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" podUID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.44:8816: connect: connection refused" Sep 09 07:10:03.224186 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:03.224142 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:10:03.224577 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:03.224211 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:10:03.224577 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:03.224233 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:10:03.224577 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:03.224252 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:10:04.224177 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:04.224138 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:10:04.224495 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:04.224477 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:10:04.224859 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:04.224602 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:10:04.225475 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:04.225447 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:10:04.225598 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:04.225510 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:10:04.225598 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:04.225544 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:10:04.229711 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:04.229686 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:10:05.171885 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:05.171852 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:10:08.895092 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:08.895023 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" podUID="213f3de8-84fc-4f01-9c00-59d846385367" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.43:8443: connect: connection refused" Sep 09 07:10:08.895556 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:08.895197 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:10:09.768108 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:09.768044 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" podUID="213f3de8-84fc-4f01-9c00-59d846385367" containerName="online" probeResult="failure" output="Get \"https://10.132.0.43:6567/health\": dial tcp 10.132.0.43:6567: connect: connection refused" Sep 09 07:10:09.768257 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:09.768188 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:10:12.201011 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:12.200977 2559 generic.go:356] "Generic (PLEG): container finished" podID="213f3de8-84fc-4f01-9c00-59d846385367" containerID="f4813ffd6e5be421709a7411348ee8f6e8066bb28b8329fbcb86bccca59fb520" exitCode=137 Sep 09 07:10:12.201011 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:12.201004 2559 generic.go:356] "Generic (PLEG): container finished" podID="213f3de8-84fc-4f01-9c00-59d846385367" containerID="a9cacdfd26ce80f2067b91e17fe52c904c83d7a6be218b3cc9a3fbd65e594fa6" exitCode=137 Sep 09 07:10:12.201462 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:12.201070 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" event={"ID":"213f3de8-84fc-4f01-9c00-59d846385367","Type":"ContainerDied","Data":"f4813ffd6e5be421709a7411348ee8f6e8066bb28b8329fbcb86bccca59fb520"} Sep 09 07:10:12.201462 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:12.201118 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" event={"ID":"213f3de8-84fc-4f01-9c00-59d846385367","Type":"ContainerDied","Data":"a9cacdfd26ce80f2067b91e17fe52c904c83d7a6be218b3cc9a3fbd65e594fa6"} Sep 09 07:10:12.539359 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:12.539337 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:10:12.631925 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:12.631899 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdp69\" (UniqueName: \"kubernetes.io/projected/213f3de8-84fc-4f01-9c00-59d846385367-kube-api-access-fdp69\") pod \"213f3de8-84fc-4f01-9c00-59d846385367\" (UID: \"213f3de8-84fc-4f01-9c00-59d846385367\") " Sep 09 07:10:12.632115 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:12.631983 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/213f3de8-84fc-4f01-9c00-59d846385367-online-tls\") pod \"213f3de8-84fc-4f01-9c00-59d846385367\" (UID: \"213f3de8-84fc-4f01-9c00-59d846385367\") " Sep 09 07:10:12.632115 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:12.632008 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/213f3de8-84fc-4f01-9c00-59d846385367-ui-tls\") pod \"213f3de8-84fc-4f01-9c00-59d846385367\" (UID: \"213f3de8-84fc-4f01-9c00-59d846385367\") " Sep 09 07:10:12.632115 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:12.632026 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/213f3de8-84fc-4f01-9c00-59d846385367-registry-tls\") pod \"213f3de8-84fc-4f01-9c00-59d846385367\" (UID: \"213f3de8-84fc-4f01-9c00-59d846385367\") " Sep 09 07:10:12.632115 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:12.632049 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/213f3de8-84fc-4f01-9c00-59d846385367-offline-tls\") pod \"213f3de8-84fc-4f01-9c00-59d846385367\" (UID: \"213f3de8-84fc-4f01-9c00-59d846385367\") " Sep 09 07:10:12.632115 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:12.632092 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/213f3de8-84fc-4f01-9c00-59d846385367-feast-data\") pod \"213f3de8-84fc-4f01-9c00-59d846385367\" (UID: \"213f3de8-84fc-4f01-9c00-59d846385367\") " Sep 09 07:10:12.632782 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:12.632740 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/213f3de8-84fc-4f01-9c00-59d846385367-feast-data" (OuterVolumeSpecName: "feast-data") pod "213f3de8-84fc-4f01-9c00-59d846385367" (UID: "213f3de8-84fc-4f01-9c00-59d846385367"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 07:10:12.634397 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:12.634373 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/213f3de8-84fc-4f01-9c00-59d846385367-online-tls" (OuterVolumeSpecName: "online-tls") pod "213f3de8-84fc-4f01-9c00-59d846385367" (UID: "213f3de8-84fc-4f01-9c00-59d846385367"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 07:10:12.634515 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:12.634436 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/213f3de8-84fc-4f01-9c00-59d846385367-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "213f3de8-84fc-4f01-9c00-59d846385367" (UID: "213f3de8-84fc-4f01-9c00-59d846385367"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 07:10:12.634515 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:12.634473 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/213f3de8-84fc-4f01-9c00-59d846385367-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "213f3de8-84fc-4f01-9c00-59d846385367" (UID: "213f3de8-84fc-4f01-9c00-59d846385367"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 07:10:12.634596 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:12.634514 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/213f3de8-84fc-4f01-9c00-59d846385367-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "213f3de8-84fc-4f01-9c00-59d846385367" (UID: "213f3de8-84fc-4f01-9c00-59d846385367"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 07:10:12.634596 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:12.634539 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/213f3de8-84fc-4f01-9c00-59d846385367-kube-api-access-fdp69" (OuterVolumeSpecName: "kube-api-access-fdp69") pod "213f3de8-84fc-4f01-9c00-59d846385367" (UID: "213f3de8-84fc-4f01-9c00-59d846385367"). InnerVolumeSpecName "kube-api-access-fdp69". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 07:10:12.732927 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:12.732849 2559 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/213f3de8-84fc-4f01-9c00-59d846385367-online-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:10:12.732927 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:12.732878 2559 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/213f3de8-84fc-4f01-9c00-59d846385367-ui-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:10:12.732927 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:12.732891 2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/213f3de8-84fc-4f01-9c00-59d846385367-registry-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:10:12.732927 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:12.732902 2559 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/213f3de8-84fc-4f01-9c00-59d846385367-offline-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:10:12.732927 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:12.732912 2559 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/213f3de8-84fc-4f01-9c00-59d846385367-feast-data\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:10:12.732927 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:12.732924 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdp69\" (UniqueName: \"kubernetes.io/projected/213f3de8-84fc-4f01-9c00-59d846385367-kube-api-access-fdp69\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:10:13.208665 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:13.208636 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" Sep 09 07:10:13.208665 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:13.208651 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg" event={"ID":"213f3de8-84fc-4f01-9c00-59d846385367","Type":"ContainerDied","Data":"72ac7826471296e1e01310ece0ba229fd31140f4e2b041147ed72759dc5374ee"} Sep 09 07:10:13.209211 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:13.208717 2559 scope.go:117] "RemoveContainer" containerID="2d3fd844e10832badac1884e9210b90aec3f4ec0de60ee2b5182a7ae9d12d557" Sep 09 07:10:13.217257 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:13.217240 2559 scope.go:117] "RemoveContainer" containerID="f4813ffd6e5be421709a7411348ee8f6e8066bb28b8329fbcb86bccca59fb520" Sep 09 07:10:13.224890 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:13.224849 2559 scope.go:117] "RemoveContainer" containerID="edaaa448d1596eec9a0675b548af6f478ba2feecc8a20ec36c9dc1fae32eb673" Sep 09 07:10:13.227297 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:13.227277 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg"] Sep 09 07:10:13.231168 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:13.231144 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-xqckg"] Sep 09 07:10:13.233387 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:13.233370 2559 scope.go:117] "RemoveContainer" containerID="a9cacdfd26ce80f2067b91e17fe52c904c83d7a6be218b3cc9a3fbd65e594fa6" Sep 09 07:10:13.240785 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:13.240764 2559 scope.go:117] "RemoveContainer" containerID="351fad40a5d68dac9fdf08fb06deedd8995c1ac6740c86f8777b147a71a23410" Sep 09 07:10:13.247925 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:13.247885 2559 scope.go:117] "RemoveContainer" containerID="e6b935efc4b0d00fa630d7bc95cf6b421d1d3761cceaf296ffd865c086baf7c6" Sep 09 07:10:14.702358 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:10:14.702315 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="213f3de8-84fc-4f01-9c00-59d846385367" path="/var/lib/kubelet/pods/213f3de8-84fc-4f01-9c00-59d846385367/volumes" Sep 09 07:11:07.021889 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.021854 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj"] Sep 09 07:11:07.022388 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.022190 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="213f3de8-84fc-4f01-9c00-59d846385367" containerName="offline" Sep 09 07:11:07.022388 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.022202 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="213f3de8-84fc-4f01-9c00-59d846385367" containerName="offline" Sep 09 07:11:07.022388 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.022215 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="213f3de8-84fc-4f01-9c00-59d846385367" containerName="registry" Sep 09 07:11:07.022388 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.022220 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="213f3de8-84fc-4f01-9c00-59d846385367" containerName="registry" Sep 09 07:11:07.022388 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.022234 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="213f3de8-84fc-4f01-9c00-59d846385367" containerName="feast-init" Sep 09 07:11:07.022388 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.022240 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="213f3de8-84fc-4f01-9c00-59d846385367" containerName="feast-init" Sep 09 07:11:07.022388 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.022245 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="213f3de8-84fc-4f01-9c00-59d846385367" containerName="feast-apply" Sep 09 07:11:07.022388 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.022251 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="213f3de8-84fc-4f01-9c00-59d846385367" containerName="feast-apply" Sep 09 07:11:07.022388 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.022258 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="213f3de8-84fc-4f01-9c00-59d846385367" containerName="ui" Sep 09 07:11:07.022388 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.022264 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="213f3de8-84fc-4f01-9c00-59d846385367" containerName="ui" Sep 09 07:11:07.022388 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.022276 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="213f3de8-84fc-4f01-9c00-59d846385367" containerName="online" Sep 09 07:11:07.022388 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.022281 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="213f3de8-84fc-4f01-9c00-59d846385367" containerName="online" Sep 09 07:11:07.022388 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.022350 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="213f3de8-84fc-4f01-9c00-59d846385367" containerName="registry" Sep 09 07:11:07.022388 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.022360 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="213f3de8-84fc-4f01-9c00-59d846385367" containerName="online" Sep 09 07:11:07.022388 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.022365 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="213f3de8-84fc-4f01-9c00-59d846385367" containerName="ui" Sep 09 07:11:07.022388 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.022372 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="213f3de8-84fc-4f01-9c00-59d846385367" containerName="offline" Sep 09 07:11:07.025674 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.025651 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:11:07.028459 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.028436 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 09 07:11:07.028598 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.028578 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 09 07:11:07.028774 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.028751 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 09 07:11:07.028907 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.028781 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 09 07:11:07.028907 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.028786 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 09 07:11:07.028907 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.028786 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 09 07:11:07.029442 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.029414 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-hvlft\"" Sep 09 07:11:07.040039 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.040015 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj"] Sep 09 07:11:07.080506 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.080483 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/44e1d998-de66-43ae-acd2-c4e951bdf383-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-nkwjj\" (UID: \"44e1d998-de66-43ae-acd2-c4e951bdf383\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:11:07.080593 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.080520 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/44e1d998-de66-43ae-acd2-c4e951bdf383-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-nkwjj\" (UID: \"44e1d998-de66-43ae-acd2-c4e951bdf383\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:11:07.080593 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.080541 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/44e1d998-de66-43ae-acd2-c4e951bdf383-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-nkwjj\" (UID: \"44e1d998-de66-43ae-acd2-c4e951bdf383\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:11:07.080593 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.080557 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t669l\" (UniqueName: \"kubernetes.io/projected/44e1d998-de66-43ae-acd2-c4e951bdf383-kube-api-access-t669l\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-nkwjj\" (UID: \"44e1d998-de66-43ae-acd2-c4e951bdf383\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:11:07.080701 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.080609 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/44e1d998-de66-43ae-acd2-c4e951bdf383-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-nkwjj\" (UID: \"44e1d998-de66-43ae-acd2-c4e951bdf383\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:11:07.080701 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.080626 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/44e1d998-de66-43ae-acd2-c4e951bdf383-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-nkwjj\" (UID: \"44e1d998-de66-43ae-acd2-c4e951bdf383\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:11:07.181486 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.181448 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/44e1d998-de66-43ae-acd2-c4e951bdf383-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-nkwjj\" (UID: \"44e1d998-de66-43ae-acd2-c4e951bdf383\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:11:07.181615 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.181505 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/44e1d998-de66-43ae-acd2-c4e951bdf383-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-nkwjj\" (UID: \"44e1d998-de66-43ae-acd2-c4e951bdf383\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:11:07.181615 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.181529 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/44e1d998-de66-43ae-acd2-c4e951bdf383-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-nkwjj\" (UID: \"44e1d998-de66-43ae-acd2-c4e951bdf383\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:11:07.181615 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.181547 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t669l\" (UniqueName: \"kubernetes.io/projected/44e1d998-de66-43ae-acd2-c4e951bdf383-kube-api-access-t669l\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-nkwjj\" (UID: \"44e1d998-de66-43ae-acd2-c4e951bdf383\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:11:07.181615 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:11:07.181574 2559 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-offline-tls: secret "feast-simple-feast-remote-setup-offline-tls" not found Sep 09 07:11:07.181781 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:11:07.181634 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44e1d998-de66-43ae-acd2-c4e951bdf383-offline-tls podName:44e1d998-de66-43ae-acd2-c4e951bdf383 nodeName:}" failed. No retries permitted until 2026-09-09 07:11:07.681618411 +0000 UTC m=+1965.578035811 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/44e1d998-de66-43ae-acd2-c4e951bdf383-offline-tls") pod "feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" (UID: "44e1d998-de66-43ae-acd2-c4e951bdf383") : secret "feast-simple-feast-remote-setup-offline-tls" not found Sep 09 07:11:07.181781 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:11:07.181651 2559 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-online-tls: secret "feast-simple-feast-remote-setup-online-tls" not found Sep 09 07:11:07.181781 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.181693 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/44e1d998-de66-43ae-acd2-c4e951bdf383-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-nkwjj\" (UID: \"44e1d998-de66-43ae-acd2-c4e951bdf383\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:11:07.181781 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:11:07.181700 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44e1d998-de66-43ae-acd2-c4e951bdf383-online-tls podName:44e1d998-de66-43ae-acd2-c4e951bdf383 nodeName:}" failed. No retries permitted until 2026-09-09 07:11:07.681684297 +0000 UTC m=+1965.578101711 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/44e1d998-de66-43ae-acd2-c4e951bdf383-online-tls") pod "feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" (UID: "44e1d998-de66-43ae-acd2-c4e951bdf383") : secret "feast-simple-feast-remote-setup-online-tls" not found Sep 09 07:11:07.181781 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.181741 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/44e1d998-de66-43ae-acd2-c4e951bdf383-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-nkwjj\" (UID: \"44e1d998-de66-43ae-acd2-c4e951bdf383\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:11:07.181986 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:11:07.181855 2559 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 09 07:11:07.181986 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:11:07.181906 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44e1d998-de66-43ae-acd2-c4e951bdf383-ui-tls podName:44e1d998-de66-43ae-acd2-c4e951bdf383 nodeName:}" failed. No retries permitted until 2026-09-09 07:11:07.681894729 +0000 UTC m=+1965.578312134 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/44e1d998-de66-43ae-acd2-c4e951bdf383-ui-tls") pod "feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" (UID: "44e1d998-de66-43ae-acd2-c4e951bdf383") : secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 09 07:11:07.182095 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.181994 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/44e1d998-de66-43ae-acd2-c4e951bdf383-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-nkwjj\" (UID: \"44e1d998-de66-43ae-acd2-c4e951bdf383\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:11:07.182245 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.182228 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/44e1d998-de66-43ae-acd2-c4e951bdf383-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-nkwjj\" (UID: \"44e1d998-de66-43ae-acd2-c4e951bdf383\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:11:07.191786 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.191764 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t669l\" (UniqueName: \"kubernetes.io/projected/44e1d998-de66-43ae-acd2-c4e951bdf383-kube-api-access-t669l\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-nkwjj\" (UID: \"44e1d998-de66-43ae-acd2-c4e951bdf383\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:11:07.687405 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.687365 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/44e1d998-de66-43ae-acd2-c4e951bdf383-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-nkwjj\" (UID: \"44e1d998-de66-43ae-acd2-c4e951bdf383\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:11:07.687603 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.687433 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/44e1d998-de66-43ae-acd2-c4e951bdf383-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-nkwjj\" (UID: \"44e1d998-de66-43ae-acd2-c4e951bdf383\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:11:07.687603 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.687483 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/44e1d998-de66-43ae-acd2-c4e951bdf383-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-nkwjj\" (UID: \"44e1d998-de66-43ae-acd2-c4e951bdf383\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:11:07.689770 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.689739 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/44e1d998-de66-43ae-acd2-c4e951bdf383-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-nkwjj\" (UID: \"44e1d998-de66-43ae-acd2-c4e951bdf383\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:11:07.689870 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.689858 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/44e1d998-de66-43ae-acd2-c4e951bdf383-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-nkwjj\" (UID: \"44e1d998-de66-43ae-acd2-c4e951bdf383\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:11:07.689996 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.689973 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/44e1d998-de66-43ae-acd2-c4e951bdf383-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-nkwjj\" (UID: \"44e1d998-de66-43ae-acd2-c4e951bdf383\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:11:07.934917 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:07.934883 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:11:08.210368 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:08.210339 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj"] Sep 09 07:11:08.212828 ip-10-0-132-137 kubenswrapper[2559]: W0909 07:11:08.212786 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod44e1d998_de66_43ae_acd2_c4e951bdf383.slice/crio-2b666f9d0525b8fec7d0ce986d1609d4be53ae3881f39ba9ed153c2a1ed71fdd WatchSource:0}: Error finding container 2b666f9d0525b8fec7d0ce986d1609d4be53ae3881f39ba9ed153c2a1ed71fdd: Status 404 returned error can't find the container with id 2b666f9d0525b8fec7d0ce986d1609d4be53ae3881f39ba9ed153c2a1ed71fdd Sep 09 07:11:08.452223 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:08.451411 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" event={"ID":"44e1d998-de66-43ae-acd2-c4e951bdf383","Type":"ContainerStarted","Data":"e859d9927e282a1045b522f31b32a63ef2a5631d4f5c885d22f21361a638de43"} Sep 09 07:11:08.452223 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:08.451467 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" event={"ID":"44e1d998-de66-43ae-acd2-c4e951bdf383","Type":"ContainerStarted","Data":"2b666f9d0525b8fec7d0ce986d1609d4be53ae3881f39ba9ed153c2a1ed71fdd"} Sep 09 07:11:13.476632 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:13.476596 2559 generic.go:356] "Generic (PLEG): container finished" podID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerID="e859d9927e282a1045b522f31b32a63ef2a5631d4f5c885d22f21361a638de43" exitCode=0 Sep 09 07:11:13.477026 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:13.476672 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" event={"ID":"44e1d998-de66-43ae-acd2-c4e951bdf383","Type":"ContainerDied","Data":"e859d9927e282a1045b522f31b32a63ef2a5631d4f5c885d22f21361a638de43"} Sep 09 07:11:14.483517 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:14.483483 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" event={"ID":"44e1d998-de66-43ae-acd2-c4e951bdf383","Type":"ContainerStarted","Data":"6e27251dd2123243912b40ab6693135aa9797f3db9ad437ad38f5936a6c8c5b0"} Sep 09 07:11:36.582279 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:36.582247 2559 generic.go:356] "Generic (PLEG): container finished" podID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerID="6e27251dd2123243912b40ab6693135aa9797f3db9ad437ad38f5936a6c8c5b0" exitCode=0 Sep 09 07:11:36.582682 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:36.582318 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" event={"ID":"44e1d998-de66-43ae-acd2-c4e951bdf383","Type":"ContainerDied","Data":"6e27251dd2123243912b40ab6693135aa9797f3db9ad437ad38f5936a6c8c5b0"} Sep 09 07:11:37.591323 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:37.591282 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" event={"ID":"44e1d998-de66-43ae-acd2-c4e951bdf383","Type":"ContainerStarted","Data":"6c587b821e022a8b3319573c6f8fdee2cf554dcf903ec9408ca4f317cdeee04a"} Sep 09 07:11:37.591758 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:37.591350 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" event={"ID":"44e1d998-de66-43ae-acd2-c4e951bdf383","Type":"ContainerStarted","Data":"43ece50011cf7adbd702e3adb1552e61206b062492739cbc28cdc19abf5343a6"} Sep 09 07:11:37.591758 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:37.591372 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" event={"ID":"44e1d998-de66-43ae-acd2-c4e951bdf383","Type":"ContainerStarted","Data":"64a85eedb4435659c358d368058ec6866aae0bdc12e9a9164d4b7433bdcaa02a"} Sep 09 07:11:37.619755 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:37.619694 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" podStartSLOduration=30.619675298 podStartE2EDuration="30.619675298s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:11:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:11:37.617334777 +0000 UTC m=+1995.513752201" watchObservedRunningTime="2026-09-09 07:11:37.619675298 +0000 UTC m=+1995.516092721" Sep 09 07:11:37.935532 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:37.935494 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:11:37.935817 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:37.935786 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:11:37.935817 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:37.935819 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:11:37.936002 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:37.935837 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:11:37.936002 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:37.935856 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:11:37.936002 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:37.935875 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:11:37.937239 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:37.937208 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" podUID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.45:8443: connect: connection refused" Sep 09 07:11:37.937340 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:37.937229 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" podUID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.45:8816: connect: connection refused" Sep 09 07:11:37.937340 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:37.937295 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" podUID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerName="online" probeResult="failure" output="Get \"https://10.132.0.45:6567/health\": dial tcp 10.132.0.45:6567: connect: connection refused" Sep 09 07:11:40.936232 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:40.936188 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" podUID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.45:8443: connect: connection refused" Sep 09 07:11:40.936786 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:40.936261 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" podUID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerName="online" probeResult="failure" output="Get \"https://10.132.0.45:6567/health\": dial tcp 10.132.0.45:6567: connect: connection refused" Sep 09 07:11:40.936786 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:40.936197 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" podUID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.45:8816: connect: connection refused" Sep 09 07:11:43.936046 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:43.935998 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" podUID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.45:8816: connect: connection refused" Sep 09 07:11:43.936563 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:43.936006 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" podUID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.45:8443: connect: connection refused" Sep 09 07:11:43.936563 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:43.936131 2559 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" podUID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerName="online" probeResult="failure" output="Get \"https://10.132.0.45:6567/health\": dial tcp 10.132.0.45:6567: connect: connection refused" Sep 09 07:11:46.936999 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:46.936967 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:11:46.937419 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:46.937108 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:11:46.937797 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:46.937769 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:11:46.938093 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:46.938077 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:11:47.761956 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:47.761926 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:11:48.606791 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:48.606760 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:11:48.940179 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:11:48.940141 2559 prober.go:120] "Probe failed" probeType="Liveness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" podUID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerName="online" probeResult="failure" output="Get \"https://10.132.0.45:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 09 07:12:08.977224 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:12:08.977174 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" podUID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerName="online" probeResult="failure" output="Get \"https://10.132.0.45:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 09 07:12:28.977223 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:12:28.977172 2559 prober.go:120] "Probe failed" probeType="Liveness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" podUID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerName="online" probeResult="failure" output="Get \"https://10.132.0.45:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 09 07:12:48.940255 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:12:48.940168 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" podUID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerName="online" probeResult="failure" output="Get \"https://10.132.0.45:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 09 07:12:51.287572 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:12:51.287540 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj"] Sep 09 07:12:51.288021 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:12:51.287772 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" podUID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerName="online" containerID="cri-o://64a85eedb4435659c358d368058ec6866aae0bdc12e9a9164d4b7433bdcaa02a" gracePeriod=30 Sep 09 07:12:51.288021 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:12:51.287828 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" podUID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerName="ui" containerID="cri-o://6c587b821e022a8b3319573c6f8fdee2cf554dcf903ec9408ca4f317cdeee04a" gracePeriod=30 Sep 09 07:12:51.288021 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:12:51.287843 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" podUID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerName="offline" containerID="cri-o://43ece50011cf7adbd702e3adb1552e61206b062492739cbc28cdc19abf5343a6" gracePeriod=30 Sep 09 07:12:51.489511 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:12:51.489468 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l"] Sep 09 07:12:51.489960 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:12:51.489760 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" podUID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerName="registry" containerID="cri-o://b03b78d6584000389b7a04a1a96fdc547fb0f8468d3bbec70346d1d63c1fba24" gracePeriod=30 Sep 09 07:12:51.489960 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:12:51.489823 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" podUID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerName="offline" containerID="cri-o://f649e8fe7419529acbd847a89d949beddec0b031599268d9b7ad78f52f3441d3" gracePeriod=30 Sep 09 07:12:51.489960 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:12:51.489846 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" podUID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerName="online" containerID="cri-o://c0e8043ac18aa9448ebbe67c770fa31f9b4a05661d8d9e482e93ed59fa8351cf" gracePeriod=30 Sep 09 07:12:51.489960 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:12:51.489857 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" podUID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerName="ui" containerID="cri-o://b9a0a997cd490f85574a6e32651216150b77bccc448cbd239eb306e79b78882e" gracePeriod=30 Sep 09 07:12:51.933979 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:12:51.933937 2559 generic.go:356] "Generic (PLEG): container finished" podID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerID="6c587b821e022a8b3319573c6f8fdee2cf554dcf903ec9408ca4f317cdeee04a" exitCode=0 Sep 09 07:12:51.934206 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:12:51.934005 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" event={"ID":"44e1d998-de66-43ae-acd2-c4e951bdf383","Type":"ContainerDied","Data":"6c587b821e022a8b3319573c6f8fdee2cf554dcf903ec9408ca4f317cdeee04a"} Sep 09 07:12:52.941326 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:12:52.941293 2559 generic.go:356] "Generic (PLEG): container finished" podID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerID="b9a0a997cd490f85574a6e32651216150b77bccc448cbd239eb306e79b78882e" exitCode=0 Sep 09 07:12:52.941326 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:12:52.941318 2559 generic.go:356] "Generic (PLEG): container finished" podID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerID="c0e8043ac18aa9448ebbe67c770fa31f9b4a05661d8d9e482e93ed59fa8351cf" exitCode=0 Sep 09 07:12:52.941765 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:12:52.941366 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" event={"ID":"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d","Type":"ContainerDied","Data":"b9a0a997cd490f85574a6e32651216150b77bccc448cbd239eb306e79b78882e"} Sep 09 07:12:52.941765 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:12:52.941410 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" event={"ID":"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d","Type":"ContainerDied","Data":"c0e8043ac18aa9448ebbe67c770fa31f9b4a05661d8d9e482e93ed59fa8351cf"} Sep 09 07:12:52.944030 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:12:52.944009 2559 generic.go:356] "Generic (PLEG): container finished" podID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerID="64a85eedb4435659c358d368058ec6866aae0bdc12e9a9164d4b7433bdcaa02a" exitCode=0 Sep 09 07:12:52.944172 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:12:52.944085 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" event={"ID":"44e1d998-de66-43ae-acd2-c4e951bdf383","Type":"ContainerDied","Data":"64a85eedb4435659c358d368058ec6866aae0bdc12e9a9164d4b7433bdcaa02a"} Sep 09 07:12:54.225912 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:12:54.225862 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" podUID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.44:8443: connect: connection refused" Sep 09 07:12:55.168971 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:12:55.168931 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" podUID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerName="online" probeResult="failure" output="Get \"https://10.132.0.44:6567/health\": dial tcp 10.132.0.44:6567: connect: connection refused" Sep 09 07:12:56.938645 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:12:56.938599 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" podUID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.45:8443: connect: connection refused" Sep 09 07:12:57.936154 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:12:57.936116 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" podUID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerName="online" probeResult="failure" output="Get \"https://10.132.0.45:6567/health\": dial tcp 10.132.0.45:6567: connect: connection refused" Sep 09 07:13:04.226113 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:04.226066 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" podUID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.44:8443: connect: connection refused" Sep 09 07:13:05.169329 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:05.169289 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" podUID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerName="online" probeResult="failure" output="Get \"https://10.132.0.44:6567/health\": dial tcp 10.132.0.44:6567: connect: connection refused" Sep 09 07:13:06.937916 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:06.937870 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" podUID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.45:8443: connect: connection refused" Sep 09 07:13:07.935447 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:07.935402 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" podUID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerName="online" probeResult="failure" output="Get \"https://10.132.0.45:6567/health\": dial tcp 10.132.0.45:6567: connect: connection refused" Sep 09 07:13:07.935632 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:07.935556 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:13:14.226093 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:14.226021 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" podUID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.44:8443: connect: connection refused" Sep 09 07:13:14.226553 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:14.226233 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:13:15.169522 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:15.169481 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" podUID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerName="online" probeResult="failure" output="Get \"https://10.132.0.44:6567/health\": dial tcp 10.132.0.44:6567: connect: connection refused" Sep 09 07:13:15.169724 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:15.169591 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:13:16.938146 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:16.938087 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" podUID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.45:8443: connect: connection refused" Sep 09 07:13:16.938624 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:16.938242 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:13:17.935933 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:17.935897 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" podUID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerName="online" probeResult="failure" output="Get \"https://10.132.0.45:6567/health\": dial tcp 10.132.0.45:6567: connect: connection refused" Sep 09 07:13:21.943039 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:21.943013 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:13:22.030766 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.030738 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/44e1d998-de66-43ae-acd2-c4e951bdf383-feast-data\") pod \"44e1d998-de66-43ae-acd2-c4e951bdf383\" (UID: \"44e1d998-de66-43ae-acd2-c4e951bdf383\") " Sep 09 07:13:22.030916 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.030778 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t669l\" (UniqueName: \"kubernetes.io/projected/44e1d998-de66-43ae-acd2-c4e951bdf383-kube-api-access-t669l\") pod \"44e1d998-de66-43ae-acd2-c4e951bdf383\" (UID: \"44e1d998-de66-43ae-acd2-c4e951bdf383\") " Sep 09 07:13:22.030916 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.030816 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/44e1d998-de66-43ae-acd2-c4e951bdf383-online-tls\") pod \"44e1d998-de66-43ae-acd2-c4e951bdf383\" (UID: \"44e1d998-de66-43ae-acd2-c4e951bdf383\") " Sep 09 07:13:22.030916 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.030880 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/44e1d998-de66-43ae-acd2-c4e951bdf383-registry-tls\") pod \"44e1d998-de66-43ae-acd2-c4e951bdf383\" (UID: \"44e1d998-de66-43ae-acd2-c4e951bdf383\") " Sep 09 07:13:22.031080 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.030918 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/44e1d998-de66-43ae-acd2-c4e951bdf383-ui-tls\") pod \"44e1d998-de66-43ae-acd2-c4e951bdf383\" (UID: \"44e1d998-de66-43ae-acd2-c4e951bdf383\") " Sep 09 07:13:22.031080 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.030954 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/44e1d998-de66-43ae-acd2-c4e951bdf383-offline-tls\") pod \"44e1d998-de66-43ae-acd2-c4e951bdf383\" (UID: \"44e1d998-de66-43ae-acd2-c4e951bdf383\") " Sep 09 07:13:22.031429 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.031374 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44e1d998-de66-43ae-acd2-c4e951bdf383-feast-data" (OuterVolumeSpecName: "feast-data") pod "44e1d998-de66-43ae-acd2-c4e951bdf383" (UID: "44e1d998-de66-43ae-acd2-c4e951bdf383"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 07:13:22.031429 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.031384 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44e1d998-de66-43ae-acd2-c4e951bdf383-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "44e1d998-de66-43ae-acd2-c4e951bdf383" (UID: "44e1d998-de66-43ae-acd2-c4e951bdf383"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 07:13:22.033178 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.033149 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44e1d998-de66-43ae-acd2-c4e951bdf383-online-tls" (OuterVolumeSpecName: "online-tls") pod "44e1d998-de66-43ae-acd2-c4e951bdf383" (UID: "44e1d998-de66-43ae-acd2-c4e951bdf383"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 07:13:22.033178 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.033170 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44e1d998-de66-43ae-acd2-c4e951bdf383-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "44e1d998-de66-43ae-acd2-c4e951bdf383" (UID: "44e1d998-de66-43ae-acd2-c4e951bdf383"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 07:13:22.033305 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.033186 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44e1d998-de66-43ae-acd2-c4e951bdf383-kube-api-access-t669l" (OuterVolumeSpecName: "kube-api-access-t669l") pod "44e1d998-de66-43ae-acd2-c4e951bdf383" (UID: "44e1d998-de66-43ae-acd2-c4e951bdf383"). InnerVolumeSpecName "kube-api-access-t669l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 07:13:22.033411 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.033396 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44e1d998-de66-43ae-acd2-c4e951bdf383-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "44e1d998-de66-43ae-acd2-c4e951bdf383" (UID: "44e1d998-de66-43ae-acd2-c4e951bdf383"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 07:13:22.070569 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.070537 2559 generic.go:356] "Generic (PLEG): container finished" podID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerID="f649e8fe7419529acbd847a89d949beddec0b031599268d9b7ad78f52f3441d3" exitCode=137 Sep 09 07:13:22.070569 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.070570 2559 generic.go:356] "Generic (PLEG): container finished" podID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerID="b03b78d6584000389b7a04a1a96fdc547fb0f8468d3bbec70346d1d63c1fba24" exitCode=137 Sep 09 07:13:22.070759 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.070611 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" event={"ID":"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d","Type":"ContainerDied","Data":"f649e8fe7419529acbd847a89d949beddec0b031599268d9b7ad78f52f3441d3"} Sep 09 07:13:22.070759 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.070661 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" event={"ID":"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d","Type":"ContainerDied","Data":"b03b78d6584000389b7a04a1a96fdc547fb0f8468d3bbec70346d1d63c1fba24"} Sep 09 07:13:22.073528 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.073504 2559 generic.go:356] "Generic (PLEG): container finished" podID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerID="43ece50011cf7adbd702e3adb1552e61206b062492739cbc28cdc19abf5343a6" exitCode=137 Sep 09 07:13:22.073642 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.073564 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" event={"ID":"44e1d998-de66-43ae-acd2-c4e951bdf383","Type":"ContainerDied","Data":"43ece50011cf7adbd702e3adb1552e61206b062492739cbc28cdc19abf5343a6"} Sep 09 07:13:22.073642 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.073581 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" Sep 09 07:13:22.073642 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.073601 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj" event={"ID":"44e1d998-de66-43ae-acd2-c4e951bdf383","Type":"ContainerDied","Data":"2b666f9d0525b8fec7d0ce986d1609d4be53ae3881f39ba9ed153c2a1ed71fdd"} Sep 09 07:13:22.073642 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.073622 2559 scope.go:117] "RemoveContainer" containerID="6c587b821e022a8b3319573c6f8fdee2cf554dcf903ec9408ca4f317cdeee04a" Sep 09 07:13:22.084694 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.084678 2559 scope.go:117] "RemoveContainer" containerID="43ece50011cf7adbd702e3adb1552e61206b062492739cbc28cdc19abf5343a6" Sep 09 07:13:22.092368 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.092352 2559 scope.go:117] "RemoveContainer" containerID="64a85eedb4435659c358d368058ec6866aae0bdc12e9a9164d4b7433bdcaa02a" Sep 09 07:13:22.097631 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.097609 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj"] Sep 09 07:13:22.101482 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.101458 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-nkwjj"] Sep 09 07:13:22.102163 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.102147 2559 scope.go:117] "RemoveContainer" containerID="6e27251dd2123243912b40ab6693135aa9797f3db9ad437ad38f5936a6c8c5b0" Sep 09 07:13:22.109448 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.109430 2559 scope.go:117] "RemoveContainer" containerID="e859d9927e282a1045b522f31b32a63ef2a5631d4f5c885d22f21361a638de43" Sep 09 07:13:22.116976 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.116949 2559 scope.go:117] "RemoveContainer" containerID="6c587b821e022a8b3319573c6f8fdee2cf554dcf903ec9408ca4f317cdeee04a" Sep 09 07:13:22.117237 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:13:22.117219 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c587b821e022a8b3319573c6f8fdee2cf554dcf903ec9408ca4f317cdeee04a\": container with ID starting with 6c587b821e022a8b3319573c6f8fdee2cf554dcf903ec9408ca4f317cdeee04a not found: ID does not exist" containerID="6c587b821e022a8b3319573c6f8fdee2cf554dcf903ec9408ca4f317cdeee04a" Sep 09 07:13:22.117300 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.117247 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c587b821e022a8b3319573c6f8fdee2cf554dcf903ec9408ca4f317cdeee04a"} err="failed to get container status \"6c587b821e022a8b3319573c6f8fdee2cf554dcf903ec9408ca4f317cdeee04a\": rpc error: code = NotFound desc = could not find container \"6c587b821e022a8b3319573c6f8fdee2cf554dcf903ec9408ca4f317cdeee04a\": container with ID starting with 6c587b821e022a8b3319573c6f8fdee2cf554dcf903ec9408ca4f317cdeee04a not found: ID does not exist" Sep 09 07:13:22.117300 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.117268 2559 scope.go:117] "RemoveContainer" containerID="43ece50011cf7adbd702e3adb1552e61206b062492739cbc28cdc19abf5343a6" Sep 09 07:13:22.117521 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:13:22.117507 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43ece50011cf7adbd702e3adb1552e61206b062492739cbc28cdc19abf5343a6\": container with ID starting with 43ece50011cf7adbd702e3adb1552e61206b062492739cbc28cdc19abf5343a6 not found: ID does not exist" containerID="43ece50011cf7adbd702e3adb1552e61206b062492739cbc28cdc19abf5343a6" Sep 09 07:13:22.117571 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.117525 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43ece50011cf7adbd702e3adb1552e61206b062492739cbc28cdc19abf5343a6"} err="failed to get container status \"43ece50011cf7adbd702e3adb1552e61206b062492739cbc28cdc19abf5343a6\": rpc error: code = NotFound desc = could not find container \"43ece50011cf7adbd702e3adb1552e61206b062492739cbc28cdc19abf5343a6\": container with ID starting with 43ece50011cf7adbd702e3adb1552e61206b062492739cbc28cdc19abf5343a6 not found: ID does not exist" Sep 09 07:13:22.117571 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.117538 2559 scope.go:117] "RemoveContainer" containerID="64a85eedb4435659c358d368058ec6866aae0bdc12e9a9164d4b7433bdcaa02a" Sep 09 07:13:22.117739 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:13:22.117724 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64a85eedb4435659c358d368058ec6866aae0bdc12e9a9164d4b7433bdcaa02a\": container with ID starting with 64a85eedb4435659c358d368058ec6866aae0bdc12e9a9164d4b7433bdcaa02a not found: ID does not exist" containerID="64a85eedb4435659c358d368058ec6866aae0bdc12e9a9164d4b7433bdcaa02a" Sep 09 07:13:22.117782 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.117741 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64a85eedb4435659c358d368058ec6866aae0bdc12e9a9164d4b7433bdcaa02a"} err="failed to get container status \"64a85eedb4435659c358d368058ec6866aae0bdc12e9a9164d4b7433bdcaa02a\": rpc error: code = NotFound desc = could not find container \"64a85eedb4435659c358d368058ec6866aae0bdc12e9a9164d4b7433bdcaa02a\": container with ID starting with 64a85eedb4435659c358d368058ec6866aae0bdc12e9a9164d4b7433bdcaa02a not found: ID does not exist" Sep 09 07:13:22.117782 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.117756 2559 scope.go:117] "RemoveContainer" containerID="6e27251dd2123243912b40ab6693135aa9797f3db9ad437ad38f5936a6c8c5b0" Sep 09 07:13:22.118015 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:13:22.118000 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e27251dd2123243912b40ab6693135aa9797f3db9ad437ad38f5936a6c8c5b0\": container with ID starting with 6e27251dd2123243912b40ab6693135aa9797f3db9ad437ad38f5936a6c8c5b0 not found: ID does not exist" containerID="6e27251dd2123243912b40ab6693135aa9797f3db9ad437ad38f5936a6c8c5b0" Sep 09 07:13:22.118075 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.118019 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e27251dd2123243912b40ab6693135aa9797f3db9ad437ad38f5936a6c8c5b0"} err="failed to get container status \"6e27251dd2123243912b40ab6693135aa9797f3db9ad437ad38f5936a6c8c5b0\": rpc error: code = NotFound desc = could not find container \"6e27251dd2123243912b40ab6693135aa9797f3db9ad437ad38f5936a6c8c5b0\": container with ID starting with 6e27251dd2123243912b40ab6693135aa9797f3db9ad437ad38f5936a6c8c5b0 not found: ID does not exist" Sep 09 07:13:22.118075 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.118032 2559 scope.go:117] "RemoveContainer" containerID="e859d9927e282a1045b522f31b32a63ef2a5631d4f5c885d22f21361a638de43" Sep 09 07:13:22.118305 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:13:22.118281 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e859d9927e282a1045b522f31b32a63ef2a5631d4f5c885d22f21361a638de43\": container with ID starting with e859d9927e282a1045b522f31b32a63ef2a5631d4f5c885d22f21361a638de43 not found: ID does not exist" containerID="e859d9927e282a1045b522f31b32a63ef2a5631d4f5c885d22f21361a638de43" Sep 09 07:13:22.118362 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.118314 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e859d9927e282a1045b522f31b32a63ef2a5631d4f5c885d22f21361a638de43"} err="failed to get container status \"e859d9927e282a1045b522f31b32a63ef2a5631d4f5c885d22f21361a638de43\": rpc error: code = NotFound desc = could not find container \"e859d9927e282a1045b522f31b32a63ef2a5631d4f5c885d22f21361a638de43\": container with ID starting with e859d9927e282a1045b522f31b32a63ef2a5631d4f5c885d22f21361a638de43 not found: ID does not exist" Sep 09 07:13:22.131664 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.131646 2559 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/44e1d998-de66-43ae-acd2-c4e951bdf383-feast-data\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:13:22.131804 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.131668 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t669l\" (UniqueName: \"kubernetes.io/projected/44e1d998-de66-43ae-acd2-c4e951bdf383-kube-api-access-t669l\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:13:22.131804 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.131707 2559 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/44e1d998-de66-43ae-acd2-c4e951bdf383-online-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:13:22.131804 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.131721 2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/44e1d998-de66-43ae-acd2-c4e951bdf383-registry-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:13:22.131804 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.131734 2559 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/44e1d998-de66-43ae-acd2-c4e951bdf383-ui-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:13:22.131804 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.131746 2559 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/44e1d998-de66-43ae-acd2-c4e951bdf383-offline-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:13:22.137273 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.137253 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:13:22.232035 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.232009 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-ui-tls\") pod \"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d\" (UID: \"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d\") " Sep 09 07:13:22.232035 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.232038 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-registry-tls\") pod \"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d\" (UID: \"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d\") " Sep 09 07:13:22.232213 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.232080 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jpdtp\" (UniqueName: \"kubernetes.io/projected/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-kube-api-access-jpdtp\") pod \"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d\" (UID: \"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d\") " Sep 09 07:13:22.232213 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.232189 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-offline-tls\") pod \"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d\" (UID: \"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d\") " Sep 09 07:13:22.232286 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.232228 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-online-tls\") pod \"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d\" (UID: \"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d\") " Sep 09 07:13:22.232286 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.232254 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-feast-data\") pod \"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d\" (UID: \"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d\") " Sep 09 07:13:22.232986 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.232952 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-feast-data" (OuterVolumeSpecName: "feast-data") pod "4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" (UID: "4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 07:13:22.234332 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.234304 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" (UID: "4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 07:13:22.234418 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.234310 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-kube-api-access-jpdtp" (OuterVolumeSpecName: "kube-api-access-jpdtp") pod "4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" (UID: "4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d"). InnerVolumeSpecName "kube-api-access-jpdtp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 07:13:22.234418 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.234314 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" (UID: "4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 07:13:22.234563 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.234546 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" (UID: "4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 07:13:22.234645 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.234626 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-online-tls" (OuterVolumeSpecName: "online-tls") pod "4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" (UID: "4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 07:13:22.333211 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.333148 2559 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-offline-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:13:22.333211 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.333174 2559 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-online-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:13:22.333211 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.333183 2559 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-feast-data\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:13:22.333211 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.333190 2559 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-ui-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:13:22.333211 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.333198 2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-registry-tls\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:13:22.333211 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.333206 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jpdtp\" (UniqueName: \"kubernetes.io/projected/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d-kube-api-access-jpdtp\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:13:22.647866 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.647773 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nf9gf_26d28788-4f98-454b-ab39-2d0d857aa674/console-operator/2.log" Sep 09 07:13:22.656858 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.656822 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nf9gf_26d28788-4f98-454b-ab39-2d0d857aa674/console-operator/2.log" Sep 09 07:13:22.702474 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:22.702433 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44e1d998-de66-43ae-acd2-c4e951bdf383" path="/var/lib/kubelet/pods/44e1d998-de66-43ae-acd2-c4e951bdf383/volumes" Sep 09 07:13:23.081898 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:23.081860 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" event={"ID":"4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d","Type":"ContainerDied","Data":"7334ccd74609596cca0c8eaba8e62dfb0d988479ad0125fbf76680faa4859768"} Sep 09 07:13:23.081898 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:23.081903 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l" Sep 09 07:13:23.082464 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:23.081924 2559 scope.go:117] "RemoveContainer" containerID="b9a0a997cd490f85574a6e32651216150b77bccc448cbd239eb306e79b78882e" Sep 09 07:13:23.090126 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:23.090103 2559 scope.go:117] "RemoveContainer" containerID="f649e8fe7419529acbd847a89d949beddec0b031599268d9b7ad78f52f3441d3" Sep 09 07:13:23.097859 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:23.097837 2559 scope.go:117] "RemoveContainer" containerID="c0e8043ac18aa9448ebbe67c770fa31f9b4a05661d8d9e482e93ed59fa8351cf" Sep 09 07:13:23.100836 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:23.100793 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l"] Sep 09 07:13:23.104666 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:23.104642 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-gx48l"] Sep 09 07:13:23.105919 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:23.105903 2559 scope.go:117] "RemoveContainer" containerID="b03b78d6584000389b7a04a1a96fdc547fb0f8468d3bbec70346d1d63c1fba24" Sep 09 07:13:23.113340 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:23.113320 2559 scope.go:117] "RemoveContainer" containerID="d622af38b96eac078bcb5ed370ec300c16f58815cdf3732b58ffb2ae24b1e782" Sep 09 07:13:23.120456 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:23.120439 2559 scope.go:117] "RemoveContainer" containerID="d7d36f04cb963185c96da0a0ba67f9390847bfc5891d959b90c003cbd81c7c04" Sep 09 07:13:24.702571 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:24.702542 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" path="/var/lib/kubelet/pods/4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d/volumes" Sep 09 07:13:40.619939 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.619899 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fp85v/must-gather-mh6k5"] Sep 09 07:13:40.620343 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.620252 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerName="feast-apply" Sep 09 07:13:40.620343 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.620266 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerName="feast-apply" Sep 09 07:13:40.620343 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.620273 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerName="registry" Sep 09 07:13:40.620343 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.620279 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerName="registry" Sep 09 07:13:40.620343 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.620284 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerName="online" Sep 09 07:13:40.620343 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.620290 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerName="online" Sep 09 07:13:40.620343 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.620297 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerName="offline" Sep 09 07:13:40.620343 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.620302 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerName="offline" Sep 09 07:13:40.620343 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.620309 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerName="ui" Sep 09 07:13:40.620343 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.620314 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerName="ui" Sep 09 07:13:40.620343 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.620323 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerName="feast-init" Sep 09 07:13:40.620343 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.620328 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerName="feast-init" Sep 09 07:13:40.620343 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.620336 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerName="ui" Sep 09 07:13:40.620343 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.620342 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerName="ui" Sep 09 07:13:40.620343 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.620348 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerName="offline" Sep 09 07:13:40.620343 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.620352 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerName="offline" Sep 09 07:13:40.620808 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.620361 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerName="feast-apply" Sep 09 07:13:40.620808 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.620366 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerName="feast-apply" Sep 09 07:13:40.620808 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.620373 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerName="online" Sep 09 07:13:40.620808 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.620379 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerName="online" Sep 09 07:13:40.620808 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.620389 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerName="feast-init" Sep 09 07:13:40.620808 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.620394 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerName="feast-init" Sep 09 07:13:40.620808 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.620441 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerName="online" Sep 09 07:13:40.620808 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.620449 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerName="ui" Sep 09 07:13:40.620808 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.620458 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerName="registry" Sep 09 07:13:40.620808 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.620463 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerName="ui" Sep 09 07:13:40.620808 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.620469 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerName="offline" Sep 09 07:13:40.620808 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.620478 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e9a3ccc-ad34-49d7-9578-7eaab7ef9c7d" containerName="offline" Sep 09 07:13:40.620808 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.620483 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="44e1d998-de66-43ae-acd2-c4e951bdf383" containerName="online" Sep 09 07:13:40.623279 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.623259 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fp85v/must-gather-mh6k5" Sep 09 07:13:40.625937 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.625916 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fp85v\"/\"openshift-service-ca.crt\"" Sep 09 07:13:40.626041 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.625941 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-fp85v\"/\"default-dockercfg-7b8zq\"" Sep 09 07:13:40.626041 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.625921 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fp85v\"/\"kube-root-ca.crt\"" Sep 09 07:13:40.632397 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.632377 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fp85v/must-gather-mh6k5"] Sep 09 07:13:40.690390 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.690369 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1e5e9d9f-34eb-4d14-a7a3-e18403451c09-must-gather-output\") pod \"must-gather-mh6k5\" (UID: \"1e5e9d9f-34eb-4d14-a7a3-e18403451c09\") " pod="openshift-must-gather-fp85v/must-gather-mh6k5" Sep 09 07:13:40.690502 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.690435 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njd4b\" (UniqueName: \"kubernetes.io/projected/1e5e9d9f-34eb-4d14-a7a3-e18403451c09-kube-api-access-njd4b\") pod \"must-gather-mh6k5\" (UID: \"1e5e9d9f-34eb-4d14-a7a3-e18403451c09\") " pod="openshift-must-gather-fp85v/must-gather-mh6k5" Sep 09 07:13:40.790964 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.790934 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njd4b\" (UniqueName: \"kubernetes.io/projected/1e5e9d9f-34eb-4d14-a7a3-e18403451c09-kube-api-access-njd4b\") pod \"must-gather-mh6k5\" (UID: \"1e5e9d9f-34eb-4d14-a7a3-e18403451c09\") " pod="openshift-must-gather-fp85v/must-gather-mh6k5" Sep 09 07:13:40.791090 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.790983 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1e5e9d9f-34eb-4d14-a7a3-e18403451c09-must-gather-output\") pod \"must-gather-mh6k5\" (UID: \"1e5e9d9f-34eb-4d14-a7a3-e18403451c09\") " pod="openshift-must-gather-fp85v/must-gather-mh6k5" Sep 09 07:13:40.791263 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.791247 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1e5e9d9f-34eb-4d14-a7a3-e18403451c09-must-gather-output\") pod \"must-gather-mh6k5\" (UID: \"1e5e9d9f-34eb-4d14-a7a3-e18403451c09\") " pod="openshift-must-gather-fp85v/must-gather-mh6k5" Sep 09 07:13:40.798174 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.798151 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njd4b\" (UniqueName: \"kubernetes.io/projected/1e5e9d9f-34eb-4d14-a7a3-e18403451c09-kube-api-access-njd4b\") pod \"must-gather-mh6k5\" (UID: \"1e5e9d9f-34eb-4d14-a7a3-e18403451c09\") " pod="openshift-must-gather-fp85v/must-gather-mh6k5" Sep 09 07:13:40.949285 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:40.949248 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fp85v/must-gather-mh6k5" Sep 09 07:13:41.215707 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:41.215633 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fp85v/must-gather-mh6k5"] Sep 09 07:13:41.218850 ip-10-0-132-137 kubenswrapper[2559]: W0909 07:13:41.218817 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1e5e9d9f_34eb_4d14_a7a3_e18403451c09.slice/crio-b4c221789b51ee0e70839129c74ede2bc478d11322a0380885cf51660511eee2 WatchSource:0}: Error finding container b4c221789b51ee0e70839129c74ede2bc478d11322a0380885cf51660511eee2: Status 404 returned error can't find the container with id b4c221789b51ee0e70839129c74ede2bc478d11322a0380885cf51660511eee2 Sep 09 07:13:41.220635 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:41.220614 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 07:13:42.165631 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:42.165552 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fp85v/must-gather-mh6k5" event={"ID":"1e5e9d9f-34eb-4d14-a7a3-e18403451c09","Type":"ContainerStarted","Data":"b4c221789b51ee0e70839129c74ede2bc478d11322a0380885cf51660511eee2"} Sep 09 07:13:51.209354 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:51.209308 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fp85v/must-gather-mh6k5" event={"ID":"1e5e9d9f-34eb-4d14-a7a3-e18403451c09","Type":"ContainerStarted","Data":"c34ef6d35584d98ea86cdcf717f3b83ae5c7742ef27e02691dc3aac5bc80b254"} Sep 09 07:13:51.209354 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:51.209357 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fp85v/must-gather-mh6k5" event={"ID":"1e5e9d9f-34eb-4d14-a7a3-e18403451c09","Type":"ContainerStarted","Data":"a375cc037640dc8d404b503c869c106db2b6fd96d1fc76d8671bd75e8fae0ed3"} Sep 09 07:13:51.225243 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:13:51.225186 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fp85v/must-gather-mh6k5" podStartSLOduration=2.090177959 podStartE2EDuration="11.225170951s" totalImagesPullingTime="9.134992992s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:13:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:13:51.22382079 +0000 UTC m=+2129.120238211" watchObservedRunningTime="2026-09-09 07:13:51.225170951 +0000 UTC m=+2129.121588373" Sep 09 07:14:00.249789 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:00.249754 2559 generic.go:356] "Generic (PLEG): container finished" podID="1e5e9d9f-34eb-4d14-a7a3-e18403451c09" containerID="a375cc037640dc8d404b503c869c106db2b6fd96d1fc76d8671bd75e8fae0ed3" exitCode=0 Sep 09 07:14:00.250263 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:00.249829 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fp85v/must-gather-mh6k5" event={"ID":"1e5e9d9f-34eb-4d14-a7a3-e18403451c09","Type":"ContainerDied","Data":"a375cc037640dc8d404b503c869c106db2b6fd96d1fc76d8671bd75e8fae0ed3"} Sep 09 07:14:00.250263 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:00.250191 2559 scope.go:117] "RemoveContainer" containerID="a375cc037640dc8d404b503c869c106db2b6fd96d1fc76d8671bd75e8fae0ed3" Sep 09 07:14:00.656017 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:00.655940 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-fp85v_must-gather-mh6k5_1e5e9d9f-34eb-4d14-a7a3-e18403451c09/gather/0.log" Sep 09 07:14:01.121839 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:01.121803 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wrmw8/must-gather-7mnq2"] Sep 09 07:14:01.124998 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:01.124966 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wrmw8/must-gather-7mnq2" Sep 09 07:14:01.129553 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:01.128228 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wrmw8\"/\"kube-root-ca.crt\"" Sep 09 07:14:01.129553 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:01.128493 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-wrmw8\"/\"default-dockercfg-8vvln\"" Sep 09 07:14:01.129553 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:01.128716 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wrmw8\"/\"openshift-service-ca.crt\"" Sep 09 07:14:01.132382 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:01.132353 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wrmw8/must-gather-7mnq2"] Sep 09 07:14:01.173981 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:01.173947 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfj2w\" (UniqueName: \"kubernetes.io/projected/61434fb9-b950-4a1a-8bad-4062142b0b26-kube-api-access-kfj2w\") pod \"must-gather-7mnq2\" (UID: \"61434fb9-b950-4a1a-8bad-4062142b0b26\") " pod="openshift-must-gather-wrmw8/must-gather-7mnq2" Sep 09 07:14:01.174169 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:01.174069 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/61434fb9-b950-4a1a-8bad-4062142b0b26-must-gather-output\") pod \"must-gather-7mnq2\" (UID: \"61434fb9-b950-4a1a-8bad-4062142b0b26\") " pod="openshift-must-gather-wrmw8/must-gather-7mnq2" Sep 09 07:14:01.274476 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:01.274440 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/61434fb9-b950-4a1a-8bad-4062142b0b26-must-gather-output\") pod \"must-gather-7mnq2\" (UID: \"61434fb9-b950-4a1a-8bad-4062142b0b26\") " pod="openshift-must-gather-wrmw8/must-gather-7mnq2" Sep 09 07:14:01.274860 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:01.274491 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfj2w\" (UniqueName: \"kubernetes.io/projected/61434fb9-b950-4a1a-8bad-4062142b0b26-kube-api-access-kfj2w\") pod \"must-gather-7mnq2\" (UID: \"61434fb9-b950-4a1a-8bad-4062142b0b26\") " pod="openshift-must-gather-wrmw8/must-gather-7mnq2" Sep 09 07:14:01.274860 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:01.274767 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/61434fb9-b950-4a1a-8bad-4062142b0b26-must-gather-output\") pod \"must-gather-7mnq2\" (UID: \"61434fb9-b950-4a1a-8bad-4062142b0b26\") " pod="openshift-must-gather-wrmw8/must-gather-7mnq2" Sep 09 07:14:01.281639 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:01.281615 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfj2w\" (UniqueName: \"kubernetes.io/projected/61434fb9-b950-4a1a-8bad-4062142b0b26-kube-api-access-kfj2w\") pod \"must-gather-7mnq2\" (UID: \"61434fb9-b950-4a1a-8bad-4062142b0b26\") " pod="openshift-must-gather-wrmw8/must-gather-7mnq2" Sep 09 07:14:01.436200 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:01.436170 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wrmw8/must-gather-7mnq2" Sep 09 07:14:01.703775 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:01.703695 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wrmw8/must-gather-7mnq2"] Sep 09 07:14:01.706768 ip-10-0-132-137 kubenswrapper[2559]: W0909 07:14:01.706741 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod61434fb9_b950_4a1a_8bad_4062142b0b26.slice/crio-e9e4a48570f926fe885fd8f8abd98cf67dc9f799cb16c2cb2cad20a1fe73ec8c WatchSource:0}: Error finding container e9e4a48570f926fe885fd8f8abd98cf67dc9f799cb16c2cb2cad20a1fe73ec8c: Status 404 returned error can't find the container with id e9e4a48570f926fe885fd8f8abd98cf67dc9f799cb16c2cb2cad20a1fe73ec8c Sep 09 07:14:02.260712 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:02.260678 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wrmw8/must-gather-7mnq2" event={"ID":"61434fb9-b950-4a1a-8bad-4062142b0b26","Type":"ContainerStarted","Data":"e9e4a48570f926fe885fd8f8abd98cf67dc9f799cb16c2cb2cad20a1fe73ec8c"} Sep 09 07:14:03.268597 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:03.268560 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wrmw8/must-gather-7mnq2" event={"ID":"61434fb9-b950-4a1a-8bad-4062142b0b26","Type":"ContainerStarted","Data":"78d95f274d0a0b22479540eaaeaeffec077ace990b98f6ba6b9a3bd85f51356c"} Sep 09 07:14:03.269035 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:03.268610 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wrmw8/must-gather-7mnq2" event={"ID":"61434fb9-b950-4a1a-8bad-4062142b0b26","Type":"ContainerStarted","Data":"9e7968f00c21db5f111364a01001e50184b53298cad3f2ccbdff4c802f482a9a"} Sep 09 07:14:03.283939 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:03.283883 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wrmw8/must-gather-7mnq2" podStartSLOduration=1.5261695560000001 podStartE2EDuration="2.283866892s" totalImagesPullingTime="757.697336ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:14:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:14:03.281959194 +0000 UTC m=+2141.178376617" watchObservedRunningTime="2026-09-09 07:14:03.283866892 +0000 UTC m=+2141.180284313" Sep 09 07:14:03.728666 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:03.728635 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-7rdlw_11156d42-6fd0-4887-9f80-6f172dcce6c1/global-pull-secret-syncer/0.log" Sep 09 07:14:03.873200 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:03.873164 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-kt6jc_3d5681e8-6984-40dc-b918-18483d2bed5c/konnectivity-agent/0.log" Sep 09 07:14:03.895407 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:03.895374 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-137.ec2.internal_323664165661da7466696aea846c55ee/haproxy/0.log" Sep 09 07:14:05.980964 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:05.980929 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-fp85v/must-gather-mh6k5"] Sep 09 07:14:05.982244 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:05.981783 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-fp85v/must-gather-mh6k5" podUID="1e5e9d9f-34eb-4d14-a7a3-e18403451c09" containerName="copy" containerID="cri-o://c34ef6d35584d98ea86cdcf717f3b83ae5c7742ef27e02691dc3aac5bc80b254" gracePeriod=2 Sep 09 07:14:05.984097 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:14:05.984047 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-mh6k5\" is forbidden: User \"system:node:ip-10-0-132-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-fp85v\": no relationship found between node 'ip-10-0-132-137.ec2.internal' and this object" podUID="1e5e9d9f-34eb-4d14-a7a3-e18403451c09" pod="openshift-must-gather-fp85v/must-gather-mh6k5" Sep 09 07:14:05.987143 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:05.987104 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-fp85v/must-gather-mh6k5"] Sep 09 07:14:06.292266 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:06.292184 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-fp85v_must-gather-mh6k5_1e5e9d9f-34eb-4d14-a7a3-e18403451c09/copy/0.log" Sep 09 07:14:06.292649 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:06.292627 2559 generic.go:356] "Generic (PLEG): container finished" podID="1e5e9d9f-34eb-4d14-a7a3-e18403451c09" containerID="c34ef6d35584d98ea86cdcf717f3b83ae5c7742ef27e02691dc3aac5bc80b254" exitCode=143 Sep 09 07:14:06.392372 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:06.391385 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-fp85v_must-gather-mh6k5_1e5e9d9f-34eb-4d14-a7a3-e18403451c09/copy/0.log" Sep 09 07:14:06.398069 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:06.397012 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fp85v/must-gather-mh6k5" Sep 09 07:14:06.399845 ip-10-0-132-137 kubenswrapper[2559]: E0909 07:14:06.399236 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-mh6k5\" is forbidden: User \"system:node:ip-10-0-132-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-fp85v\": no relationship found between node 'ip-10-0-132-137.ec2.internal' and this object" podUID="1e5e9d9f-34eb-4d14-a7a3-e18403451c09" pod="openshift-must-gather-fp85v/must-gather-mh6k5" Sep 09 07:14:06.429331 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:06.429291 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1e5e9d9f-34eb-4d14-a7a3-e18403451c09-must-gather-output\") pod \"1e5e9d9f-34eb-4d14-a7a3-e18403451c09\" (UID: \"1e5e9d9f-34eb-4d14-a7a3-e18403451c09\") " Sep 09 07:14:06.429505 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:06.429363 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njd4b\" (UniqueName: \"kubernetes.io/projected/1e5e9d9f-34eb-4d14-a7a3-e18403451c09-kube-api-access-njd4b\") pod \"1e5e9d9f-34eb-4d14-a7a3-e18403451c09\" (UID: \"1e5e9d9f-34eb-4d14-a7a3-e18403451c09\") " Sep 09 07:14:06.430382 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:06.430208 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e5e9d9f-34eb-4d14-a7a3-e18403451c09-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "1e5e9d9f-34eb-4d14-a7a3-e18403451c09" (UID: "1e5e9d9f-34eb-4d14-a7a3-e18403451c09"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 07:14:06.437203 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:06.436965 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e5e9d9f-34eb-4d14-a7a3-e18403451c09-kube-api-access-njd4b" (OuterVolumeSpecName: "kube-api-access-njd4b") pod "1e5e9d9f-34eb-4d14-a7a3-e18403451c09" (UID: "1e5e9d9f-34eb-4d14-a7a3-e18403451c09"). InnerVolumeSpecName "kube-api-access-njd4b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 07:14:06.530995 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:06.530923 2559 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1e5e9d9f-34eb-4d14-a7a3-e18403451c09-must-gather-output\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:14:06.530995 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:06.530966 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njd4b\" (UniqueName: \"kubernetes.io/projected/1e5e9d9f-34eb-4d14-a7a3-e18403451c09-kube-api-access-njd4b\") on node \"ip-10-0-132-137.ec2.internal\" DevicePath \"\"" Sep 09 07:14:06.706085 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:06.706036 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e5e9d9f-34eb-4d14-a7a3-e18403451c09" path="/var/lib/kubelet/pods/1e5e9d9f-34eb-4d14-a7a3-e18403451c09/volumes" Sep 09 07:14:07.302564 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:07.302482 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-fp85v_must-gather-mh6k5_1e5e9d9f-34eb-4d14-a7a3-e18403451c09/copy/0.log" Sep 09 07:14:07.304096 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:07.304008 2559 scope.go:117] "RemoveContainer" containerID="c34ef6d35584d98ea86cdcf717f3b83ae5c7742ef27e02691dc3aac5bc80b254" Sep 09 07:14:07.304454 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:07.304334 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fp85v/must-gather-mh6k5" Sep 09 07:14:07.328398 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:07.328371 2559 scope.go:117] "RemoveContainer" containerID="a375cc037640dc8d404b503c869c106db2b6fd96d1fc76d8671bd75e8fae0ed3" Sep 09 07:14:07.855581 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:07.855551 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e4040508-57f2-404e-87bf-0406cd190e56/alertmanager/0.log" Sep 09 07:14:07.879085 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:07.879044 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e4040508-57f2-404e-87bf-0406cd190e56/config-reloader/0.log" Sep 09 07:14:07.900169 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:07.900135 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e4040508-57f2-404e-87bf-0406cd190e56/kube-rbac-proxy-web/0.log" Sep 09 07:14:07.921026 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:07.920988 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e4040508-57f2-404e-87bf-0406cd190e56/kube-rbac-proxy/0.log" Sep 09 07:14:07.942866 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:07.942827 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e4040508-57f2-404e-87bf-0406cd190e56/kube-rbac-proxy-metric/0.log" Sep 09 07:14:07.964195 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:07.964161 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e4040508-57f2-404e-87bf-0406cd190e56/prom-label-proxy/0.log" Sep 09 07:14:07.984942 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:07.984909 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e4040508-57f2-404e-87bf-0406cd190e56/init-config-reloader/0.log" Sep 09 07:14:08.025236 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:08.025197 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-cflhw_39fe02da-f881-42af-9eb7-35cc7e1a0c66/cluster-monitoring-operator/0.log" Sep 09 07:14:08.362762 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:08.362729 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xjl8t_11a1d01b-12f3-4e32-a60e-f0a9d8e2367a/node-exporter/0.log" Sep 09 07:14:08.387967 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:08.387929 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xjl8t_11a1d01b-12f3-4e32-a60e-f0a9d8e2367a/kube-rbac-proxy/0.log" Sep 09 07:14:08.408627 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:08.408591 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xjl8t_11a1d01b-12f3-4e32-a60e-f0a9d8e2367a/init-textfile/0.log" Sep 09 07:14:08.708487 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:08.708452 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-wbww5_8094df83-28f6-4432-94a7-80928df7d921/prometheus-operator/0.log" Sep 09 07:14:08.729026 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:08.728991 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-wbww5_8094df83-28f6-4432-94a7-80928df7d921/kube-rbac-proxy/0.log" Sep 09 07:14:08.751366 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:08.751335 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-sx9jv_43e1fde4-bef2-4106-afdd-585668465ba3/prometheus-operator-admission-webhook/0.log" Sep 09 07:14:08.780924 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:08.780892 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-546fff794d-2vkft_8987da44-4b1d-4a54-8081-1a70de01ed92/telemeter-client/0.log" Sep 09 07:14:08.801654 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:08.801624 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-546fff794d-2vkft_8987da44-4b1d-4a54-8081-1a70de01ed92/reload/0.log" Sep 09 07:14:08.827392 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:08.827352 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-546fff794d-2vkft_8987da44-4b1d-4a54-8081-1a70de01ed92/kube-rbac-proxy/0.log" Sep 09 07:14:08.857996 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:08.857968 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78b989bfc-cl4hl_694195a6-fedc-4d83-9cfd-bef2072219b9/thanos-query/0.log" Sep 09 07:14:08.883998 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:08.883963 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78b989bfc-cl4hl_694195a6-fedc-4d83-9cfd-bef2072219b9/kube-rbac-proxy-web/0.log" Sep 09 07:14:08.917496 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:08.917461 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78b989bfc-cl4hl_694195a6-fedc-4d83-9cfd-bef2072219b9/kube-rbac-proxy/0.log" Sep 09 07:14:08.945483 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:08.945441 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78b989bfc-cl4hl_694195a6-fedc-4d83-9cfd-bef2072219b9/prom-label-proxy/0.log" Sep 09 07:14:08.973646 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:08.973558 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78b989bfc-cl4hl_694195a6-fedc-4d83-9cfd-bef2072219b9/kube-rbac-proxy-rules/0.log" Sep 09 07:14:08.998187 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:08.998152 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78b989bfc-cl4hl_694195a6-fedc-4d83-9cfd-bef2072219b9/kube-rbac-proxy-metrics/0.log" Sep 09 07:14:10.191490 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:10.191462 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-5x9qz_635a2ebb-6998-49e2-8e4d-088725c2c079/networking-console-plugin/0.log" Sep 09 07:14:10.669077 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:10.668983 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nf9gf_26d28788-4f98-454b-ab39-2d0d857aa674/console-operator/2.log" Sep 09 07:14:10.675209 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:10.675180 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nf9gf_26d28788-4f98-454b-ab39-2d0d857aa674/console-operator/3.log" Sep 09 07:14:11.036108 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:11.036080 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-69f4dbf49b-j8kmp_714ac9f9-377b-4a71-8824-b6e8a1ca225b/console/0.log" Sep 09 07:14:11.065586 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:11.065555 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-4zgn6_f1089629-f49d-4225-8f67-332ea17548ca/download-server/0.log" Sep 09 07:14:11.492430 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:11.492399 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-4dkcv_89e331e0-4596-4390-9b74-b4ddce540b8a/volume-data-source-validator/0.log" Sep 09 07:14:12.186639 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:12.186600 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-g54h7_5e0761f7-70d1-4952-b645-1b99f46e3fc7/dns/0.log" Sep 09 07:14:12.195108 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:12.195080 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wrmw8/perf-node-gather-daemonset-vf7f5"] Sep 09 07:14:12.195446 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:12.195432 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e5e9d9f-34eb-4d14-a7a3-e18403451c09" containerName="gather" Sep 09 07:14:12.195489 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:12.195448 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e5e9d9f-34eb-4d14-a7a3-e18403451c09" containerName="gather" Sep 09 07:14:12.195489 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:12.195464 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e5e9d9f-34eb-4d14-a7a3-e18403451c09" containerName="copy" Sep 09 07:14:12.195489 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:12.195473 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e5e9d9f-34eb-4d14-a7a3-e18403451c09" containerName="copy" Sep 09 07:14:12.195589 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:12.195538 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e5e9d9f-34eb-4d14-a7a3-e18403451c09" containerName="gather" Sep 09 07:14:12.195589 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:12.195552 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e5e9d9f-34eb-4d14-a7a3-e18403451c09" containerName="copy" Sep 09 07:14:12.198698 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:12.198676 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wrmw8/perf-node-gather-daemonset-vf7f5" Sep 09 07:14:12.206043 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:12.206017 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wrmw8/perf-node-gather-daemonset-vf7f5"] Sep 09 07:14:12.217485 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:12.217460 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-g54h7_5e0761f7-70d1-4952-b645-1b99f46e3fc7/kube-rbac-proxy/0.log" Sep 09 07:14:12.288071 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:12.288025 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b85466b2-ffb0-429f-a520-7f4caa6ef1f8-proc\") pod \"perf-node-gather-daemonset-vf7f5\" (UID: \"b85466b2-ffb0-429f-a520-7f4caa6ef1f8\") " pod="openshift-must-gather-wrmw8/perf-node-gather-daemonset-vf7f5" Sep 09 07:14:12.288244 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:12.288148 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bzvz\" (UniqueName: \"kubernetes.io/projected/b85466b2-ffb0-429f-a520-7f4caa6ef1f8-kube-api-access-7bzvz\") pod \"perf-node-gather-daemonset-vf7f5\" (UID: \"b85466b2-ffb0-429f-a520-7f4caa6ef1f8\") " pod="openshift-must-gather-wrmw8/perf-node-gather-daemonset-vf7f5" Sep 09 07:14:12.288244 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:12.288182 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b85466b2-ffb0-429f-a520-7f4caa6ef1f8-podres\") pod \"perf-node-gather-daemonset-vf7f5\" (UID: \"b85466b2-ffb0-429f-a520-7f4caa6ef1f8\") " pod="openshift-must-gather-wrmw8/perf-node-gather-daemonset-vf7f5" Sep 09 07:14:12.288244 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:12.288225 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b85466b2-ffb0-429f-a520-7f4caa6ef1f8-lib-modules\") pod \"perf-node-gather-daemonset-vf7f5\" (UID: \"b85466b2-ffb0-429f-a520-7f4caa6ef1f8\") " pod="openshift-must-gather-wrmw8/perf-node-gather-daemonset-vf7f5" Sep 09 07:14:12.288379 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:12.288275 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b85466b2-ffb0-429f-a520-7f4caa6ef1f8-sys\") pod \"perf-node-gather-daemonset-vf7f5\" (UID: \"b85466b2-ffb0-429f-a520-7f4caa6ef1f8\") " pod="openshift-must-gather-wrmw8/perf-node-gather-daemonset-vf7f5" Sep 09 07:14:12.373595 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:12.373560 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bk8ww_0f25c33f-2f5d-4645-990b-b9d2f201e1fc/dns-node-resolver/0.log" Sep 09 07:14:12.389133 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:12.389102 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bzvz\" (UniqueName: \"kubernetes.io/projected/b85466b2-ffb0-429f-a520-7f4caa6ef1f8-kube-api-access-7bzvz\") pod \"perf-node-gather-daemonset-vf7f5\" (UID: \"b85466b2-ffb0-429f-a520-7f4caa6ef1f8\") " pod="openshift-must-gather-wrmw8/perf-node-gather-daemonset-vf7f5" Sep 09 07:14:12.389295 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:12.389150 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b85466b2-ffb0-429f-a520-7f4caa6ef1f8-podres\") pod \"perf-node-gather-daemonset-vf7f5\" (UID: \"b85466b2-ffb0-429f-a520-7f4caa6ef1f8\") " pod="openshift-must-gather-wrmw8/perf-node-gather-daemonset-vf7f5" Sep 09 07:14:12.389295 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:12.389197 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b85466b2-ffb0-429f-a520-7f4caa6ef1f8-lib-modules\") pod \"perf-node-gather-daemonset-vf7f5\" (UID: \"b85466b2-ffb0-429f-a520-7f4caa6ef1f8\") " pod="openshift-must-gather-wrmw8/perf-node-gather-daemonset-vf7f5" Sep 09 07:14:12.389295 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:12.389237 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b85466b2-ffb0-429f-a520-7f4caa6ef1f8-sys\") pod \"perf-node-gather-daemonset-vf7f5\" (UID: \"b85466b2-ffb0-429f-a520-7f4caa6ef1f8\") " pod="openshift-must-gather-wrmw8/perf-node-gather-daemonset-vf7f5" Sep 09 07:14:12.389295 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:12.389274 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b85466b2-ffb0-429f-a520-7f4caa6ef1f8-proc\") pod \"perf-node-gather-daemonset-vf7f5\" (UID: \"b85466b2-ffb0-429f-a520-7f4caa6ef1f8\") " pod="openshift-must-gather-wrmw8/perf-node-gather-daemonset-vf7f5" Sep 09 07:14:12.389666 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:12.389641 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b85466b2-ffb0-429f-a520-7f4caa6ef1f8-proc\") pod \"perf-node-gather-daemonset-vf7f5\" (UID: \"b85466b2-ffb0-429f-a520-7f4caa6ef1f8\") " pod="openshift-must-gather-wrmw8/perf-node-gather-daemonset-vf7f5" Sep 09 07:14:12.389779 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:12.389653 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b85466b2-ffb0-429f-a520-7f4caa6ef1f8-sys\") pod \"perf-node-gather-daemonset-vf7f5\" (UID: \"b85466b2-ffb0-429f-a520-7f4caa6ef1f8\") " pod="openshift-must-gather-wrmw8/perf-node-gather-daemonset-vf7f5" Sep 09 07:14:12.389779 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:12.389763 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b85466b2-ffb0-429f-a520-7f4caa6ef1f8-lib-modules\") pod \"perf-node-gather-daemonset-vf7f5\" (UID: \"b85466b2-ffb0-429f-a520-7f4caa6ef1f8\") " pod="openshift-must-gather-wrmw8/perf-node-gather-daemonset-vf7f5" Sep 09 07:14:12.389903 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:12.389763 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b85466b2-ffb0-429f-a520-7f4caa6ef1f8-podres\") pod \"perf-node-gather-daemonset-vf7f5\" (UID: \"b85466b2-ffb0-429f-a520-7f4caa6ef1f8\") " pod="openshift-must-gather-wrmw8/perf-node-gather-daemonset-vf7f5" Sep 09 07:14:12.397523 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:12.397498 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bzvz\" (UniqueName: \"kubernetes.io/projected/b85466b2-ffb0-429f-a520-7f4caa6ef1f8-kube-api-access-7bzvz\") pod \"perf-node-gather-daemonset-vf7f5\" (UID: \"b85466b2-ffb0-429f-a520-7f4caa6ef1f8\") " pod="openshift-must-gather-wrmw8/perf-node-gather-daemonset-vf7f5" Sep 09 07:14:12.513199 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:12.513121 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wrmw8/perf-node-gather-daemonset-vf7f5" Sep 09 07:14:12.796269 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:12.796167 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wrmw8/perf-node-gather-daemonset-vf7f5"] Sep 09 07:14:12.798945 ip-10-0-132-137 kubenswrapper[2559]: W0909 07:14:12.798919 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb85466b2_ffb0_429f_a520_7f4caa6ef1f8.slice/crio-c3e94b6af40247ae3d169e0fff9c166e3355a4bb42b6e5f1cc4abec1c3a203aa WatchSource:0}: Error finding container c3e94b6af40247ae3d169e0fff9c166e3355a4bb42b6e5f1cc4abec1c3a203aa: Status 404 returned error can't find the container with id c3e94b6af40247ae3d169e0fff9c166e3355a4bb42b6e5f1cc4abec1c3a203aa Sep 09 07:14:12.823599 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:12.823576 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qr5vp_f1a72723-5d93-402c-8ec3-768531d2a883/node-ca/0.log" Sep 09 07:14:13.337695 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:13.337649 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wrmw8/perf-node-gather-daemonset-vf7f5" event={"ID":"b85466b2-ffb0-429f-a520-7f4caa6ef1f8","Type":"ContainerStarted","Data":"0f4cb2cdffefc7b08f7c26573d9b914a657059d32792b8bc20f8847d22fb420f"} Sep 09 07:14:13.337896 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:13.337731 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wrmw8/perf-node-gather-daemonset-vf7f5" event={"ID":"b85466b2-ffb0-429f-a520-7f4caa6ef1f8","Type":"ContainerStarted","Data":"c3e94b6af40247ae3d169e0fff9c166e3355a4bb42b6e5f1cc4abec1c3a203aa"} Sep 09 07:14:13.337896 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:13.337763 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-wrmw8/perf-node-gather-daemonset-vf7f5" Sep 09 07:14:13.356765 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:13.356711 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wrmw8/perf-node-gather-daemonset-vf7f5" podStartSLOduration=1.3566963699999999 podStartE2EDuration="1.35669637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 07:14:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:14:13.35461704 +0000 UTC m=+2151.251034460" watchObservedRunningTime="2026-09-09 07:14:13.35669637 +0000 UTC m=+2151.253113789" Sep 09 07:14:13.736077 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:13.736030 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4lksd_7e3c731f-3b9b-445d-a6d9-38c20b30568e/serve-healthcheck-canary/0.log" Sep 09 07:14:14.094322 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:14.094214 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-lw25n_781bd79e-3b6d-493f-a5e1-aaeecb110ffd/insights-operator/0.log" Sep 09 07:14:14.094764 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:14.094743 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-lw25n_781bd79e-3b6d-493f-a5e1-aaeecb110ffd/insights-operator/1.log" Sep 09 07:14:14.246197 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:14.246162 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s4mlq_93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9/kube-rbac-proxy/0.log" Sep 09 07:14:14.265304 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:14.265277 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s4mlq_93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9/exporter/0.log" Sep 09 07:14:14.284884 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:14.284855 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s4mlq_93c58a55-9f3b-4b5d-a43b-4ff70ba6f3b9/extractor/0.log" Sep 09 07:14:18.136467 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:18.136391 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-r456s_8bb61eca-5639-4fd6-bc8e-9807a6ac290f/migrator/0.log" Sep 09 07:14:18.160258 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:18.160223 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-r456s_8bb61eca-5639-4fd6-bc8e-9807a6ac290f/graceful-termination/0.log" Sep 09 07:14:19.185758 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:19.185722 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-27vp7_13c06be2-78fb-4ceb-83ee-8cc6a545b395/kube-multus-additional-cni-plugins/0.log" Sep 09 07:14:19.207275 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:19.207243 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-27vp7_13c06be2-78fb-4ceb-83ee-8cc6a545b395/egress-router-binary-copy/0.log" Sep 09 07:14:19.229025 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:19.229002 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-27vp7_13c06be2-78fb-4ceb-83ee-8cc6a545b395/cni-plugins/0.log" Sep 09 07:14:19.251828 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:19.251806 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-27vp7_13c06be2-78fb-4ceb-83ee-8cc6a545b395/bond-cni-plugin/0.log" Sep 09 07:14:19.273310 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:19.273287 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-27vp7_13c06be2-78fb-4ceb-83ee-8cc6a545b395/routeoverride-cni/0.log" Sep 09 07:14:19.299561 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:19.299537 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-27vp7_13c06be2-78fb-4ceb-83ee-8cc6a545b395/whereabouts-cni-bincopy/0.log" Sep 09 07:14:19.320928 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:19.320904 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-27vp7_13c06be2-78fb-4ceb-83ee-8cc6a545b395/whereabouts-cni/0.log" Sep 09 07:14:19.354665 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:19.354644 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-wrmw8/perf-node-gather-daemonset-vf7f5" Sep 09 07:14:19.661798 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:19.661725 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-pfbsb_aeac7db8-f053-4218-9e15-de29565b7826/kube-multus/0.log" Sep 09 07:14:19.777307 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:19.777274 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-pkv72_336552c8-ca0f-434a-8353-bc61ea118fa5/network-metrics-daemon/0.log" Sep 09 07:14:19.794294 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:19.794254 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-pkv72_336552c8-ca0f-434a-8353-bc61ea118fa5/kube-rbac-proxy/0.log" Sep 09 07:14:20.971250 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:20.971220 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snmsv_a101aee5-3bb2-4c35-a963-a498c6c86d91/ovn-controller/0.log" Sep 09 07:14:20.996848 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:20.996790 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snmsv_a101aee5-3bb2-4c35-a963-a498c6c86d91/ovn-acl-logging/0.log" Sep 09 07:14:21.029833 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:21.029801 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snmsv_a101aee5-3bb2-4c35-a963-a498c6c86d91/kube-rbac-proxy-node/0.log" Sep 09 07:14:21.054736 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:21.054700 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snmsv_a101aee5-3bb2-4c35-a963-a498c6c86d91/kube-rbac-proxy-ovn-metrics/0.log" Sep 09 07:14:21.086452 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:21.086426 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snmsv_a101aee5-3bb2-4c35-a963-a498c6c86d91/northd/0.log" Sep 09 07:14:21.119124 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:21.119098 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snmsv_a101aee5-3bb2-4c35-a963-a498c6c86d91/nbdb/0.log" Sep 09 07:14:21.161854 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:21.161827 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snmsv_a101aee5-3bb2-4c35-a963-a498c6c86d91/sbdb/0.log" Sep 09 07:14:21.225874 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:21.225841 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snmsv_a101aee5-3bb2-4c35-a963-a498c6c86d91/ovnkube-controller/0.log" Sep 09 07:14:22.299742 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:22.299713 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-685456655f-rq4hm_b25571a8-6f7a-4395-8357-3df81376bbff/check-endpoints/0.log" Sep 09 07:14:22.322487 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:22.322463 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-9f6j7_3355475e-185e-4637-900a-5bf30d29d199/network-check-target-container/0.log" Sep 09 07:14:23.229490 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:23.229462 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-5qd9z_9c0c8eea-7803-45cf-a327-2d4c39c42360/iptables-alerter/0.log" Sep 09 07:14:23.860895 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:23.860858 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-w776r_e251251d-5f0e-4372-ada0-e70551d79b78/tuned/0.log" Sep 09 07:14:26.282737 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:26.282613 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-service-ca-operator_service-ca-operator-859f94c4b-sqtkx_dbf282a5-cce3-489a-93df-44e7b10ef6fd/service-ca-operator/1.log" Sep 09 07:14:26.284214 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:26.284132 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-service-ca-operator_service-ca-operator-859f94c4b-sqtkx_dbf282a5-cce3-489a-93df-44e7b10ef6fd/service-ca-operator/0.log" Sep 09 07:14:27.124939 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:27.124901 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-9dd8h_e2ab2571-e051-4e8e-baf4-ce549dbd71ba/csi-driver/0.log" Sep 09 07:14:27.147769 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:27.147741 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-9dd8h_e2ab2571-e051-4e8e-baf4-ce549dbd71ba/csi-node-driver-registrar/0.log" Sep 09 07:14:27.169041 ip-10-0-132-137 kubenswrapper[2559]: I0909 07:14:27.169016 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-9dd8h_e2ab2571-e051-4e8e-baf4-ce549dbd71ba/csi-liveness-probe/0.log"