Sep 08 10:06:55.850687 ip-10-0-138-137 systemd[1]: Starting Kubernetes Kubelet... Sep 08 10:06:56.201219 ip-10-0-138-137 kubenswrapper[2568]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 08 10:06:56.201219 ip-10-0-138-137 kubenswrapper[2568]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 08 10:06:56.201219 ip-10-0-138-137 kubenswrapper[2568]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 08 10:06:56.201219 ip-10-0-138-137 kubenswrapper[2568]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 08 10:06:56.201219 ip-10-0-138-137 kubenswrapper[2568]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 08 10:06:56.203061 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.202989 2568 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 08 10:06:56.206615 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206602 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 10:06:56.206615 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206616 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 10:06:56.206681 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206620 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 10:06:56.206681 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206623 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 10:06:56.206681 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206627 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 10:06:56.206681 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206630 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 10:06:56.206681 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206632 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 10:06:56.206681 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206635 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 10:06:56.206681 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206638 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 10:06:56.206681 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206640 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 10:06:56.206681 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206643 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 10:06:56.206681 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206645 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 10:06:56.206681 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206653 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 10:06:56.206681 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206655 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 10:06:56.206681 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206658 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 10:06:56.206681 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206661 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 10:06:56.206681 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206663 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 10:06:56.206681 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206665 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 10:06:56.206681 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206668 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 10:06:56.206681 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206670 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 10:06:56.206681 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206673 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 10:06:56.207157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206677 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 10:06:56.207157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206681 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 10:06:56.207157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206684 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 10:06:56.207157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206687 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 10:06:56.207157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206691 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 10:06:56.207157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206695 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 10:06:56.207157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206697 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 10:06:56.207157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206700 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 10:06:56.207157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206703 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 10:06:56.207157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206705 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 10:06:56.207157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206708 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 10:06:56.207157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206710 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 10:06:56.207157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206713 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 10:06:56.207157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206716 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 10:06:56.207157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206719 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 10:06:56.207157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206721 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 10:06:56.207157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206723 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 10:06:56.207157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206726 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 10:06:56.207157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206728 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 10:06:56.207625 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206731 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 10:06:56.207625 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206733 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 10:06:56.207625 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206735 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 10:06:56.207625 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206738 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 10:06:56.207625 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206740 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 10:06:56.207625 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206743 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 10:06:56.207625 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206745 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 10:06:56.207625 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206748 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 10:06:56.207625 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206750 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 10:06:56.207625 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206752 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 10:06:56.207625 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206754 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 10:06:56.207625 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206757 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 10:06:56.207625 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206760 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 10:06:56.207625 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206762 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 10:06:56.207625 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206765 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 10:06:56.207625 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206768 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 10:06:56.207625 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206771 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 10:06:56.207625 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206773 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 10:06:56.207625 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206776 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 10:06:56.208080 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206778 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 10:06:56.208080 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206780 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 10:06:56.208080 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206783 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 10:06:56.208080 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206785 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 10:06:56.208080 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206788 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 10:06:56.208080 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206790 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 10:06:56.208080 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206793 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 10:06:56.208080 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206795 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 10:06:56.208080 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206797 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 10:06:56.208080 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206800 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 10:06:56.208080 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206802 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 10:06:56.208080 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206805 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 10:06:56.208080 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206808 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 10:06:56.208080 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206811 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 10:06:56.208080 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206813 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 10:06:56.208080 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206816 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 10:06:56.208080 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206818 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 10:06:56.208080 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206820 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 10:06:56.208080 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206823 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 10:06:56.208080 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206825 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 10:06:56.208575 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206828 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 10:06:56.208575 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206830 2568 feature_gate.go:328] unrecognized feature gate: Example Sep 08 10:06:56.208575 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206832 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 10:06:56.208575 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206835 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 10:06:56.208575 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206841 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 10:06:56.208575 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206844 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 10:06:56.208575 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206847 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 10:06:56.208575 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206850 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 10:06:56.208575 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206854 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 10:06:56.208575 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206856 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 10:06:56.208575 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206859 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 10:06:56.208575 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206862 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 10:06:56.208575 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206864 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 10:06:56.208575 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206867 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 10:06:56.208575 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206870 2568 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 10:06:56.208575 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206872 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 10:06:56.208575 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206875 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 10:06:56.208575 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206877 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 10:06:56.208575 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206880 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 10:06:56.208575 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.206882 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 10:06:56.209067 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207546 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 10:06:56.209067 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207552 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 10:06:56.209067 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207554 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 10:06:56.209067 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207557 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 10:06:56.209067 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207560 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 10:06:56.209067 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207563 2568 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 10:06:56.209067 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207566 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 10:06:56.209067 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207568 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 10:06:56.209067 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207572 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 10:06:56.209067 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207574 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 10:06:56.209067 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207577 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 10:06:56.209067 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207580 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 10:06:56.209067 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207582 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 10:06:56.209067 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207585 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 10:06:56.209067 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207587 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 10:06:56.209067 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207590 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 10:06:56.209067 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207594 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 10:06:56.209067 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207597 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 10:06:56.209067 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207600 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 10:06:56.209067 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207603 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 10:06:56.209573 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207606 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 10:06:56.209573 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207608 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 10:06:56.209573 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207613 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 10:06:56.209573 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207617 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 10:06:56.209573 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207620 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 10:06:56.209573 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207622 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 10:06:56.209573 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207625 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 10:06:56.209573 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207628 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 10:06:56.209573 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207630 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 10:06:56.209573 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207633 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 10:06:56.209573 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207636 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 10:06:56.209573 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207638 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 10:06:56.209573 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207641 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 10:06:56.209573 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207643 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 10:06:56.209573 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207646 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 10:06:56.209573 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207648 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 10:06:56.209573 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207651 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 10:06:56.209573 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207654 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 10:06:56.209573 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207662 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 10:06:56.209573 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207665 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 10:06:56.210060 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207668 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 10:06:56.210060 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207671 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 10:06:56.210060 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207673 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 10:06:56.210060 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207675 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 10:06:56.210060 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207678 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 10:06:56.210060 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207680 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 10:06:56.210060 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207683 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 10:06:56.210060 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207685 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 10:06:56.210060 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207689 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 10:06:56.210060 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207692 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 10:06:56.210060 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207694 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 10:06:56.210060 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207697 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 10:06:56.210060 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207699 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 10:06:56.210060 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207702 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 10:06:56.210060 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207705 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 10:06:56.210060 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207707 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 10:06:56.210060 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207710 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 10:06:56.210060 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207713 2568 feature_gate.go:328] unrecognized feature gate: Example Sep 08 10:06:56.210060 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207715 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 10:06:56.210060 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207718 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 10:06:56.210554 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207720 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 10:06:56.210554 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207722 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 10:06:56.210554 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207725 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 10:06:56.210554 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207727 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 10:06:56.210554 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207730 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 10:06:56.210554 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207732 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 10:06:56.210554 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207734 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 10:06:56.210554 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207737 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 10:06:56.210554 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207739 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 10:06:56.210554 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207742 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 10:06:56.210554 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207744 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 10:06:56.210554 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207747 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 10:06:56.210554 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207750 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 10:06:56.210554 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207753 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 10:06:56.210554 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207755 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 10:06:56.210554 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207760 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 10:06:56.210554 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207763 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 10:06:56.210554 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207766 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 10:06:56.210554 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207769 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 10:06:56.211038 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207772 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 10:06:56.211038 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207775 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 10:06:56.211038 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207778 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 10:06:56.211038 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207780 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 10:06:56.211038 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207783 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 10:06:56.211038 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207785 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 10:06:56.211038 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207788 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 10:06:56.211038 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207791 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 10:06:56.211038 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207793 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 10:06:56.211038 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207796 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 10:06:56.211038 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207798 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 10:06:56.211038 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207801 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 10:06:56.211038 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207803 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 10:06:56.211038 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207806 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 10:06:56.211038 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207809 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 10:06:56.211038 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207813 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 10:06:56.211038 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207815 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 10:06:56.211038 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207818 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 10:06:56.211038 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207820 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 10:06:56.211038 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.207823 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 10:06:56.211535 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208178 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 10:06:56.211535 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208183 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 10:06:56.211535 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208185 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 10:06:56.211535 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208188 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 10:06:56.211535 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208191 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 10:06:56.211535 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208195 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 10:06:56.211535 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208197 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 10:06:56.211535 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208200 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 10:06:56.211535 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208202 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 10:06:56.211535 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208205 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 10:06:56.211535 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208207 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 10:06:56.211535 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208209 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 10:06:56.211535 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208212 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 10:06:56.211535 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208215 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 10:06:56.211535 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208218 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 10:06:56.211535 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208220 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 10:06:56.211535 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208223 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 10:06:56.211535 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208225 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 10:06:56.211535 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208228 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 10:06:56.211535 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208230 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 10:06:56.212115 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208233 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 10:06:56.212115 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208235 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 10:06:56.212115 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208238 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 10:06:56.212115 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208240 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 10:06:56.212115 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208243 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 10:06:56.212115 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208245 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 10:06:56.212115 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208248 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 10:06:56.212115 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208250 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 10:06:56.212115 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208254 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 10:06:56.212115 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208257 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 10:06:56.212115 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208260 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 10:06:56.212115 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208263 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 10:06:56.212115 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208266 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 10:06:56.212115 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208269 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 10:06:56.212115 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208271 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 10:06:56.212115 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208274 2568 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 10:06:56.212115 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208276 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 10:06:56.212115 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208280 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 10:06:56.212115 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208282 2568 feature_gate.go:328] unrecognized feature gate: Example Sep 08 10:06:56.212115 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208284 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 10:06:56.212658 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208287 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 10:06:56.212658 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208289 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 10:06:56.212658 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208292 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 10:06:56.212658 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208294 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 10:06:56.212658 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208297 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 10:06:56.212658 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208300 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 10:06:56.212658 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208302 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 10:06:56.212658 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208305 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 10:06:56.212658 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208307 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 10:06:56.212658 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208310 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 10:06:56.212658 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208312 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 10:06:56.212658 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208314 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 10:06:56.212658 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208317 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 10:06:56.212658 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208319 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 10:06:56.212658 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208322 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 10:06:56.212658 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208325 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 10:06:56.212658 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208327 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 10:06:56.212658 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208329 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 10:06:56.212658 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208332 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 10:06:56.212658 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208334 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 10:06:56.213157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208337 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 10:06:56.213157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208339 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 10:06:56.213157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208341 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 10:06:56.213157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208344 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 10:06:56.213157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208346 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 10:06:56.213157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208349 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 10:06:56.213157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208351 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 10:06:56.213157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208353 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 10:06:56.213157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208358 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 10:06:56.213157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208360 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 10:06:56.213157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208363 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 10:06:56.213157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208365 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 10:06:56.213157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208368 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 10:06:56.213157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208370 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 10:06:56.213157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208373 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 10:06:56.213157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208375 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 10:06:56.213157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208378 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 10:06:56.213157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208381 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 10:06:56.213157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208384 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 10:06:56.213157 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208386 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 10:06:56.213685 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208389 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 10:06:56.213685 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208391 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 10:06:56.213685 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208394 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 10:06:56.213685 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208396 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 10:06:56.213685 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208399 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 10:06:56.213685 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208401 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 10:06:56.213685 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208405 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 10:06:56.213685 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208423 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 10:06:56.213685 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208426 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 10:06:56.213685 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208428 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 10:06:56.213685 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208431 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 10:06:56.213685 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208433 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 10:06:56.213685 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208436 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 10:06:56.213685 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208438 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 10:06:56.213685 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208442 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 10:06:56.213685 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208446 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 10:06:56.213685 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208449 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 10:06:56.213685 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208451 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 10:06:56.213685 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.208454 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 10:06:56.214132 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208523 2568 flags.go:64] FLAG: --address="0.0.0.0" Sep 08 10:06:56.214132 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208554 2568 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 08 10:06:56.214132 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208561 2568 flags.go:64] FLAG: --anonymous-auth="true" Sep 08 10:06:56.214132 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208565 2568 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 08 10:06:56.214132 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208570 2568 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 08 10:06:56.214132 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208573 2568 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 08 10:06:56.214132 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208577 2568 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 08 10:06:56.214132 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208581 2568 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 08 10:06:56.214132 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208584 2568 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 08 10:06:56.214132 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208587 2568 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 08 10:06:56.214132 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208592 2568 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 08 10:06:56.214132 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208595 2568 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 08 10:06:56.214132 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208598 2568 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 08 10:06:56.214132 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208601 2568 flags.go:64] FLAG: --cgroup-root="" Sep 08 10:06:56.214132 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208603 2568 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 08 10:06:56.214132 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208606 2568 flags.go:64] FLAG: --client-ca-file="" Sep 08 10:06:56.214132 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208609 2568 flags.go:64] FLAG: --cloud-provider="external" Sep 08 10:06:56.214132 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208612 2568 flags.go:64] FLAG: --cluster-dns="[]" Sep 08 10:06:56.214132 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208616 2568 flags.go:64] FLAG: --cluster-domain="" Sep 08 10:06:56.214132 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208619 2568 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 08 10:06:56.214132 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208621 2568 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 08 10:06:56.214132 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208624 2568 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 08 10:06:56.214132 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208628 2568 flags.go:64] FLAG: --container-log-max-files="5" Sep 08 10:06:56.214132 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208631 2568 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 08 10:06:56.214723 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208634 2568 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 08 10:06:56.214723 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208637 2568 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 08 10:06:56.214723 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208640 2568 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 08 10:06:56.214723 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208642 2568 flags.go:64] FLAG: --contention-profiling="false" Sep 08 10:06:56.214723 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208645 2568 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 08 10:06:56.214723 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208648 2568 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 08 10:06:56.214723 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208651 2568 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 08 10:06:56.214723 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208654 2568 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 08 10:06:56.214723 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208657 2568 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 08 10:06:56.214723 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208661 2568 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 08 10:06:56.214723 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208664 2568 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 08 10:06:56.214723 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208667 2568 flags.go:64] FLAG: --enable-load-reader="false" Sep 08 10:06:56.214723 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208670 2568 flags.go:64] FLAG: --enable-server="true" Sep 08 10:06:56.214723 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208673 2568 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 08 10:06:56.214723 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208678 2568 flags.go:64] FLAG: --event-burst="100" Sep 08 10:06:56.214723 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208681 2568 flags.go:64] FLAG: --event-qps="50" Sep 08 10:06:56.214723 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208683 2568 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 08 10:06:56.214723 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208686 2568 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 08 10:06:56.214723 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208690 2568 flags.go:64] FLAG: --eviction-hard="" Sep 08 10:06:56.214723 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208693 2568 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 08 10:06:56.214723 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208696 2568 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 08 10:06:56.214723 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208699 2568 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 08 10:06:56.214723 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208701 2568 flags.go:64] FLAG: --eviction-soft="" Sep 08 10:06:56.214723 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208704 2568 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 08 10:06:56.214723 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208707 2568 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 08 10:06:56.215321 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208710 2568 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 08 10:06:56.215321 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208713 2568 flags.go:64] FLAG: --experimental-mounter-path="" Sep 08 10:06:56.215321 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208716 2568 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 08 10:06:56.215321 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208718 2568 flags.go:64] FLAG: --fail-swap-on="true" Sep 08 10:06:56.215321 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208721 2568 flags.go:64] FLAG: --feature-gates="" Sep 08 10:06:56.215321 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208725 2568 flags.go:64] FLAG: --file-check-frequency="20s" Sep 08 10:06:56.215321 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208728 2568 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 08 10:06:56.215321 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208731 2568 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 08 10:06:56.215321 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208734 2568 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 08 10:06:56.215321 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208736 2568 flags.go:64] FLAG: --healthz-port="10248" Sep 08 10:06:56.215321 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208739 2568 flags.go:64] FLAG: --help="false" Sep 08 10:06:56.215321 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208742 2568 flags.go:64] FLAG: --hostname-override="ip-10-0-138-137.ec2.internal" Sep 08 10:06:56.215321 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208745 2568 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 08 10:06:56.215321 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208747 2568 flags.go:64] FLAG: --http-check-frequency="20s" Sep 08 10:06:56.215321 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208750 2568 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 08 10:06:56.215321 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208753 2568 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 08 10:06:56.215321 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208758 2568 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 08 10:06:56.215321 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208761 2568 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 08 10:06:56.215321 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208763 2568 flags.go:64] FLAG: --image-service-endpoint="" Sep 08 10:06:56.215321 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208766 2568 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 08 10:06:56.215321 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208769 2568 flags.go:64] FLAG: --kube-api-burst="100" Sep 08 10:06:56.215321 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208772 2568 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 08 10:06:56.215321 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208775 2568 flags.go:64] FLAG: --kube-api-qps="50" Sep 08 10:06:56.215321 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208778 2568 flags.go:64] FLAG: --kube-reserved="" Sep 08 10:06:56.215933 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208780 2568 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 08 10:06:56.215933 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208785 2568 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 08 10:06:56.215933 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208788 2568 flags.go:64] FLAG: --kubelet-cgroups="" Sep 08 10:06:56.215933 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208790 2568 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 08 10:06:56.215933 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208793 2568 flags.go:64] FLAG: --lock-file="" Sep 08 10:06:56.215933 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208796 2568 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 08 10:06:56.215933 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208799 2568 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 08 10:06:56.215933 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208802 2568 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 08 10:06:56.215933 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208807 2568 flags.go:64] FLAG: --log-json-split-stream="false" Sep 08 10:06:56.215933 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208809 2568 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 08 10:06:56.215933 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208812 2568 flags.go:64] FLAG: --log-text-split-stream="false" Sep 08 10:06:56.215933 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208815 2568 flags.go:64] FLAG: --logging-format="text" Sep 08 10:06:56.215933 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208817 2568 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 08 10:06:56.215933 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208821 2568 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 08 10:06:56.215933 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208823 2568 flags.go:64] FLAG: --manifest-url="" Sep 08 10:06:56.215933 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208826 2568 flags.go:64] FLAG: --manifest-url-header="" Sep 08 10:06:56.215933 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208830 2568 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 08 10:06:56.215933 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208833 2568 flags.go:64] FLAG: --max-open-files="1000000" Sep 08 10:06:56.215933 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208837 2568 flags.go:64] FLAG: --max-pods="110" Sep 08 10:06:56.215933 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208840 2568 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 08 10:06:56.215933 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208842 2568 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 08 10:06:56.215933 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208845 2568 flags.go:64] FLAG: --memory-manager-policy="None" Sep 08 10:06:56.215933 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208848 2568 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 08 10:06:56.215933 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208851 2568 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 08 10:06:56.215933 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208854 2568 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 08 10:06:56.216595 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208867 2568 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 08 10:06:56.216595 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208874 2568 flags.go:64] FLAG: --node-status-max-images="50" Sep 08 10:06:56.216595 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208877 2568 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 08 10:06:56.216595 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208880 2568 flags.go:64] FLAG: --oom-score-adj="-999" Sep 08 10:06:56.216595 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208883 2568 flags.go:64] FLAG: --pod-cidr="" Sep 08 10:06:56.216595 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208886 2568 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 08 10:06:56.216595 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208892 2568 flags.go:64] FLAG: --pod-manifest-path="" Sep 08 10:06:56.216595 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208895 2568 flags.go:64] FLAG: --pod-max-pids="-1" Sep 08 10:06:56.216595 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208898 2568 flags.go:64] FLAG: --pods-per-core="0" Sep 08 10:06:56.216595 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208902 2568 flags.go:64] FLAG: --port="10250" Sep 08 10:06:56.216595 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208905 2568 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 08 10:06:56.216595 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208908 2568 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-09ffa72acd5383f78" Sep 08 10:06:56.216595 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208911 2568 flags.go:64] FLAG: --qos-reserved="" Sep 08 10:06:56.216595 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208914 2568 flags.go:64] FLAG: --read-only-port="10255" Sep 08 10:06:56.216595 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208917 2568 flags.go:64] FLAG: --register-node="true" Sep 08 10:06:56.216595 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208920 2568 flags.go:64] FLAG: --register-with-taints="" Sep 08 10:06:56.216595 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208923 2568 flags.go:64] FLAG: --registry-burst="10" Sep 08 10:06:56.216595 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208926 2568 flags.go:64] FLAG: --registry-qps="5" Sep 08 10:06:56.216595 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208929 2568 flags.go:64] FLAG: --reserved-cpus="" Sep 08 10:06:56.216595 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208931 2568 flags.go:64] FLAG: --reserved-memory="" Sep 08 10:06:56.216595 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208940 2568 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 08 10:06:56.216595 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208943 2568 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 08 10:06:56.216595 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208946 2568 flags.go:64] FLAG: --rotate-certificates="false" Sep 08 10:06:56.216595 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208949 2568 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 08 10:06:56.216595 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208952 2568 flags.go:64] FLAG: --runonce="false" Sep 08 10:06:56.217226 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208955 2568 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 08 10:06:56.217226 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208957 2568 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 08 10:06:56.217226 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208960 2568 flags.go:64] FLAG: --seccomp-default="false" Sep 08 10:06:56.217226 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208963 2568 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 08 10:06:56.217226 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208966 2568 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 08 10:06:56.217226 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208970 2568 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 08 10:06:56.217226 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208974 2568 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 08 10:06:56.217226 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208977 2568 flags.go:64] FLAG: --storage-driver-password="root" Sep 08 10:06:56.217226 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208979 2568 flags.go:64] FLAG: --storage-driver-secure="false" Sep 08 10:06:56.217226 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208982 2568 flags.go:64] FLAG: --storage-driver-table="stats" Sep 08 10:06:56.217226 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208985 2568 flags.go:64] FLAG: --storage-driver-user="root" Sep 08 10:06:56.217226 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208988 2568 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 08 10:06:56.217226 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208991 2568 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 08 10:06:56.217226 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208994 2568 flags.go:64] FLAG: --system-cgroups="" Sep 08 10:06:56.217226 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.208997 2568 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 08 10:06:56.217226 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.209002 2568 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 08 10:06:56.217226 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.209005 2568 flags.go:64] FLAG: --tls-cert-file="" Sep 08 10:06:56.217226 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.209008 2568 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 08 10:06:56.217226 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.209012 2568 flags.go:64] FLAG: --tls-min-version="" Sep 08 10:06:56.217226 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.209015 2568 flags.go:64] FLAG: --tls-private-key-file="" Sep 08 10:06:56.217226 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.209018 2568 flags.go:64] FLAG: --topology-manager-policy="none" Sep 08 10:06:56.217226 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.209021 2568 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 08 10:06:56.217226 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.209023 2568 flags.go:64] FLAG: --topology-manager-scope="container" Sep 08 10:06:56.217226 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.209026 2568 flags.go:64] FLAG: --v="2" Sep 08 10:06:56.217226 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.209030 2568 flags.go:64] FLAG: --version="false" Sep 08 10:06:56.217846 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.209033 2568 flags.go:64] FLAG: --vmodule="" Sep 08 10:06:56.217846 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.209037 2568 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 08 10:06:56.217846 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.209040 2568 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 08 10:06:56.217846 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209167 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 10:06:56.217846 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209172 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 10:06:56.217846 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209175 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 10:06:56.217846 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209178 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 10:06:56.217846 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209181 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 10:06:56.217846 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209184 2568 feature_gate.go:328] unrecognized feature gate: Example Sep 08 10:06:56.217846 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209187 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 10:06:56.217846 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209191 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 10:06:56.217846 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209195 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 10:06:56.217846 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209210 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 10:06:56.217846 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209215 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 10:06:56.217846 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209218 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 10:06:56.217846 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209220 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 10:06:56.217846 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209223 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 10:06:56.217846 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209225 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 10:06:56.217846 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209228 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 10:06:56.217846 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209231 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 10:06:56.218330 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209233 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 10:06:56.218330 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209239 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 10:06:56.218330 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209242 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 10:06:56.218330 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209245 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 10:06:56.218330 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209248 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 10:06:56.218330 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209250 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 10:06:56.218330 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209252 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 10:06:56.218330 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209255 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 10:06:56.218330 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209257 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 10:06:56.218330 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209260 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 10:06:56.218330 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209262 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 10:06:56.218330 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209265 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 10:06:56.218330 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209267 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 10:06:56.218330 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209270 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 10:06:56.218330 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209272 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 10:06:56.218330 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209274 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 10:06:56.218330 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209277 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 10:06:56.218330 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209280 2568 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 10:06:56.218330 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209282 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 10:06:56.218330 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209284 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 10:06:56.218828 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209286 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 10:06:56.218828 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209290 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 10:06:56.218828 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209294 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 10:06:56.218828 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209297 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 10:06:56.218828 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209299 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 10:06:56.218828 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209303 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 10:06:56.218828 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209305 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 10:06:56.218828 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209307 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 10:06:56.218828 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209310 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 10:06:56.218828 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209312 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 10:06:56.218828 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209314 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 10:06:56.218828 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209317 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 10:06:56.218828 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209320 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 10:06:56.218828 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209322 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 10:06:56.218828 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209325 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 10:06:56.218828 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209329 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 10:06:56.218828 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209332 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 10:06:56.218828 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209335 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 10:06:56.218828 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209338 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 10:06:56.219291 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209341 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 10:06:56.219291 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209344 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 10:06:56.219291 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209346 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 10:06:56.219291 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209349 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 10:06:56.219291 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209352 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 10:06:56.219291 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209354 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 10:06:56.219291 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209356 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 10:06:56.219291 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209359 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 10:06:56.219291 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209361 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 10:06:56.219291 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209364 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 10:06:56.219291 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209366 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 10:06:56.219291 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209368 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 10:06:56.219291 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209371 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 10:06:56.219291 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209373 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 10:06:56.219291 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209375 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 10:06:56.219291 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209378 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 10:06:56.219291 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209380 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 10:06:56.219291 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209383 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 10:06:56.219291 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209385 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 10:06:56.219291 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209388 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 10:06:56.219799 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209390 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 10:06:56.219799 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209392 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 10:06:56.219799 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209395 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 10:06:56.219799 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209397 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 10:06:56.219799 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209400 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 10:06:56.219799 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209402 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 10:06:56.219799 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209405 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 10:06:56.219799 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209421 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 10:06:56.219799 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209424 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 10:06:56.219799 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209426 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 10:06:56.219799 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209429 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 10:06:56.219799 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209433 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 10:06:56.219799 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209436 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 10:06:56.219799 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209438 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 10:06:56.219799 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209441 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 10:06:56.219799 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209444 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 10:06:56.219799 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209446 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 10:06:56.219799 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209449 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 10:06:56.219799 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209452 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 10:06:56.220245 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209454 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 10:06:56.220245 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209457 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 10:06:56.220245 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209459 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 10:06:56.220245 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.209461 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 10:06:56.220245 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.210048 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 08 10:06:56.220245 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.217151 2568 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 08 10:06:56.220245 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.217167 2568 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 08 10:06:56.220245 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.217937 2568 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 08 10:06:56.220245 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.217945 2568 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 08 10:06:56.220245 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218011 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 10:06:56.220245 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218016 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 10:06:56.220245 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218021 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 10:06:56.220245 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218024 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 10:06:56.220245 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218027 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 10:06:56.220245 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218030 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 10:06:56.220672 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218032 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 10:06:56.220672 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218035 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 10:06:56.220672 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218037 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 10:06:56.220672 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218040 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 10:06:56.220672 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218043 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 10:06:56.220672 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218046 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 10:06:56.220672 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218048 2568 feature_gate.go:328] unrecognized feature gate: Example Sep 08 10:06:56.220672 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218051 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 10:06:56.220672 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218054 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 10:06:56.220672 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218056 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 10:06:56.220672 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218059 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 10:06:56.220672 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218062 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 10:06:56.220672 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218064 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 10:06:56.220672 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218067 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 10:06:56.220672 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218070 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 10:06:56.220672 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218072 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 10:06:56.220672 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218075 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 10:06:56.220672 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218077 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 10:06:56.220672 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218080 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 10:06:56.220672 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218083 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 10:06:56.221201 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218086 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 10:06:56.221201 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218088 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 10:06:56.221201 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218091 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 10:06:56.221201 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218095 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 10:06:56.221201 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218099 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 10:06:56.221201 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218102 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 10:06:56.221201 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218104 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 10:06:56.221201 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218107 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 10:06:56.221201 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218110 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 10:06:56.221201 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218113 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 10:06:56.221201 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218115 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 10:06:56.221201 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218118 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 10:06:56.221201 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218120 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 10:06:56.221201 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218122 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 10:06:56.221201 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218125 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 10:06:56.221201 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218127 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 10:06:56.221201 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218130 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 10:06:56.221201 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218133 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 10:06:56.221201 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218135 2568 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 10:06:56.221201 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218138 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 10:06:56.222042 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218140 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 10:06:56.222042 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218143 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 10:06:56.222042 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218146 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 10:06:56.222042 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218148 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 10:06:56.222042 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218151 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 10:06:56.222042 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218153 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 10:06:56.222042 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218156 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 10:06:56.222042 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218158 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 10:06:56.222042 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218160 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 10:06:56.222042 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218163 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 10:06:56.222042 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218165 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 10:06:56.222042 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218168 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 10:06:56.222042 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218170 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 10:06:56.222042 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218173 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 10:06:56.222042 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218175 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 10:06:56.222042 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218179 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 10:06:56.222042 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218182 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 10:06:56.222042 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218186 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 10:06:56.222042 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218190 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 10:06:56.222649 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218193 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 10:06:56.222649 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218196 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 10:06:56.222649 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218199 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 10:06:56.222649 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218202 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 10:06:56.222649 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218204 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 10:06:56.222649 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218207 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 10:06:56.222649 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218210 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 10:06:56.222649 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218212 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 10:06:56.222649 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218215 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 10:06:56.222649 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218218 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 10:06:56.222649 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218220 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 10:06:56.222649 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218223 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 10:06:56.222649 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218226 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 10:06:56.222649 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218228 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 10:06:56.222649 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218231 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 10:06:56.222649 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218233 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 10:06:56.222649 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218236 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 10:06:56.222649 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218238 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 10:06:56.222649 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218241 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 10:06:56.222649 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218243 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 10:06:56.223135 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218247 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 10:06:56.223135 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218250 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 10:06:56.223135 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218254 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 10:06:56.223135 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218257 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 10:06:56.223135 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218260 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 10:06:56.223135 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218263 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 10:06:56.223135 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218266 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 10:06:56.223135 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218269 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 10:06:56.223135 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218272 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 10:06:56.223135 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218274 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 10:06:56.223135 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218277 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 10:06:56.223135 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218280 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 10:06:56.223135 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218282 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 10:06:56.223135 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218285 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 10:06:56.223572 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.218290 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 08 10:06:56.223572 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218405 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 10:06:56.223572 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218424 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 10:06:56.223572 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218428 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 10:06:56.223572 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218431 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 10:06:56.223572 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218434 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 10:06:56.223572 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218437 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 10:06:56.223572 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218440 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 10:06:56.223572 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218442 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 10:06:56.223572 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218445 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 10:06:56.223572 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218448 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 10:06:56.223572 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218451 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 10:06:56.223572 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218453 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 10:06:56.223572 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218456 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 10:06:56.223572 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218458 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 10:06:56.223944 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218461 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 10:06:56.223944 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218463 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 10:06:56.223944 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218466 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 10:06:56.223944 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218468 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 10:06:56.223944 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218471 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 10:06:56.223944 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218474 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 10:06:56.223944 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218476 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 10:06:56.223944 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218479 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 10:06:56.223944 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218481 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 10:06:56.223944 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218484 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 10:06:56.223944 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218486 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 10:06:56.223944 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218488 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 10:06:56.223944 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218491 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 10:06:56.223944 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218494 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 10:06:56.223944 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218504 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 10:06:56.223944 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218508 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 10:06:56.223944 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218510 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 10:06:56.223944 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218513 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 10:06:56.223944 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218515 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 10:06:56.223944 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218518 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 10:06:56.224429 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218520 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 10:06:56.224429 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218523 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 10:06:56.224429 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218525 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 10:06:56.224429 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218528 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 10:06:56.224429 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218531 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 10:06:56.224429 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218533 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 10:06:56.224429 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218536 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 10:06:56.224429 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218538 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 10:06:56.224429 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218541 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 10:06:56.224429 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218543 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 10:06:56.224429 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218546 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 10:06:56.224429 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218549 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 10:06:56.224429 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218553 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 10:06:56.224429 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218556 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 10:06:56.224429 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218558 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 10:06:56.224429 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218562 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 10:06:56.224429 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218566 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 10:06:56.224429 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218569 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 10:06:56.224429 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218571 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 10:06:56.224894 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218574 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 10:06:56.224894 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218577 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 10:06:56.224894 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218580 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 10:06:56.224894 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218582 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 10:06:56.224894 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218585 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 10:06:56.224894 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218587 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 10:06:56.224894 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218590 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 10:06:56.224894 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218593 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 10:06:56.224894 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218596 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 10:06:56.224894 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218598 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 10:06:56.224894 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218601 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 10:06:56.224894 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218603 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 10:06:56.224894 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218606 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 10:06:56.224894 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218608 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 10:06:56.224894 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218611 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 10:06:56.224894 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218613 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 10:06:56.224894 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218616 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 10:06:56.224894 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218618 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 10:06:56.224894 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218621 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 10:06:56.224894 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218623 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 10:06:56.225365 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218626 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 10:06:56.225365 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218628 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 10:06:56.225365 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218631 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 10:06:56.225365 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218634 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 10:06:56.225365 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218636 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 10:06:56.225365 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218638 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 10:06:56.225365 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218641 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 10:06:56.225365 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218644 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 10:06:56.225365 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218646 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 10:06:56.225365 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218648 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 10:06:56.225365 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218651 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 10:06:56.225365 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218653 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 10:06:56.225365 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218656 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 10:06:56.225365 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218658 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 10:06:56.225365 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218661 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 10:06:56.225365 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218663 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 10:06:56.225365 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218665 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 10:06:56.225365 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218668 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 10:06:56.225365 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218671 2568 feature_gate.go:328] unrecognized feature gate: Example Sep 08 10:06:56.225365 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218674 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 10:06:56.225866 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218678 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 10:06:56.225866 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218681 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 10:06:56.225866 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218683 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 10:06:56.225866 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218686 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 10:06:56.225866 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218689 2568 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 10:06:56.225866 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:56.218691 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 10:06:56.225866 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.218696 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 08 10:06:56.225866 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.218794 2568 server.go:962] "Client rotation is on, will bootstrap in background" Sep 08 10:06:56.225866 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.220931 2568 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 08 10:06:56.225866 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.223314 2568 server.go:1019] "Starting client certificate rotation" Sep 08 10:06:56.225866 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.223400 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 08 10:06:56.225866 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.223494 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 08 10:06:56.226590 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.226561 2568 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 08 10:06:56.227638 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.227622 2568 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 08 10:06:56.237661 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.237641 2568 log.go:25] "Validated CRI v1 runtime API" Sep 08 10:06:56.242279 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.242267 2568 log.go:25] "Validated CRI v1 image API" Sep 08 10:06:56.243116 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.243100 2568 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 08 10:06:56.246503 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.246486 2568 fs.go:135] Filesystem UUIDs: map[60ce010c-dea0-4315-8b7f-7b88792f8e16:/dev/nvme0n1p4 6de6ef26-8b08-4fe9-8eb5-af9724316924:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Sep 08 10:06:56.246582 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.246502 2568 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 08 10:06:56.249770 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.249751 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 08 10:06:56.251580 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.251481 2568 manager.go:216] Machine: {Timestamp:2026-09-08 10:06:56.251000371 +0000 UTC m=+0.295104445 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3104396 MemoryCapacity:32812163072 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28768f8a207b4e0f4fa29dfdb96b1f SystemUUID:ec28768f-8a20-7b4e-0f4f-a29dfdb96b1f BootID:3758d104-f91c-44b8-af7d-6d0c085d2f24 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:92:9c:0a:82:d9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:92:9c:0a:82:d9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1a:a5:36:b7:9d:de Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812163072 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 08 10:06:56.251657 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.251593 2568 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 08 10:06:56.251722 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.251659 2568 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 08 10:06:56.254810 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.254784 2568 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 08 10:06:56.254931 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.254813 2568 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-137.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 08 10:06:56.255535 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.255526 2568 topology_manager.go:138] "Creating topology manager with none policy" Sep 08 10:06:56.255566 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.255538 2568 container_manager_linux.go:309] "Creating device plugin manager" Sep 08 10:06:56.255566 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.255546 2568 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 08 10:06:56.256158 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.256148 2568 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 08 10:06:56.257202 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.257191 2568 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 08 10:06:56.262085 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.262073 2568 state_mem.go:36] "Initialized new in-memory state store" Sep 08 10:06:56.262196 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.262187 2568 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 08 10:06:56.262229 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.262217 2568 kubelet.go:486] "Attempting to sync node with API server" Sep 08 10:06:56.262256 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.262230 2568 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 08 10:06:56.262256 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.262244 2568 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 08 10:06:56.262256 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.262253 2568 kubelet.go:398] "Adding apiserver pod source" Sep 08 10:06:56.262350 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.262261 2568 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 08 10:06:56.262604 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.262587 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 08 10:06:56.262697 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.262614 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 08 10:06:56.264670 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.264637 2568 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 08 10:06:56.267569 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.267551 2568 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 08 10:06:56.267655 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.267583 2568 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 08 10:06:56.268734 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.268720 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 08 10:06:56.268734 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.268737 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 08 10:06:56.268834 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.268742 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 08 10:06:56.268834 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.268748 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 08 10:06:56.268834 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.268756 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 08 10:06:56.268834 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.268761 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 08 10:06:56.268834 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.268774 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 08 10:06:56.268834 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.268779 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 08 10:06:56.268834 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.268785 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 08 10:06:56.268834 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.268791 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 08 10:06:56.268834 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.268798 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 08 10:06:56.268834 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.268803 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 08 10:06:56.269525 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.269506 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 08 10:06:56.269525 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.269515 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 08 10:06:56.271721 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.271698 2568 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-137.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 08 10:06:56.271819 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:56.271738 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 08 10:06:56.271869 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:56.271812 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-137.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 08 10:06:56.273202 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.273187 2568 server.go:1268] "Started kubelet" Sep 08 10:06:56.273334 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.273286 2568 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 08 10:06:56.273376 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.273358 2568 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 08 10:06:56.273658 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.273256 2568 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 08 10:06:56.274057 ip-10-0-138-137 systemd[1]: Started Kubernetes Kubelet. Sep 08 10:06:56.274346 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.274200 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jnpv5" Sep 08 10:06:56.275486 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.275471 2568 server.go:310] "Adding debug handlers to kubelet server" Sep 08 10:06:56.277468 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.277450 2568 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 08 10:06:56.277558 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:56.277541 2568 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 08 10:06:56.280538 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.280521 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jnpv5" Sep 08 10:06:56.283385 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.283368 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 08 10:06:56.283928 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.283911 2568 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 08 10:06:56.284173 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:56.284152 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-137.ec2.internal\" not found" Sep 08 10:06:56.284487 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.284471 2568 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 08 10:06:56.284487 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.284485 2568 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 08 10:06:56.284611 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.284499 2568 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 08 10:06:56.284657 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.284610 2568 reconstruct.go:97] "Volume reconstruction finished" Sep 08 10:06:56.284657 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.284620 2568 reconciler.go:29] "Reconciler: start to sync state" Sep 08 10:06:56.284975 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.284960 2568 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 08 10:06:56.285026 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.284978 2568 factory.go:55] Registering systemd factory Sep 08 10:06:56.285026 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.284987 2568 factory.go:223] Registration of the systemd container factory successfully Sep 08 10:06:56.285205 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.285189 2568 factory.go:153] Registering CRI-O factory Sep 08 10:06:56.285205 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.285206 2568 factory.go:223] Registration of the crio container factory successfully Sep 08 10:06:56.285350 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.285225 2568 factory.go:103] Registering Raw factory Sep 08 10:06:56.285350 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.285239 2568 manager.go:1196] Started watching for new ooms in manager Sep 08 10:06:56.285634 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.285616 2568 manager.go:349] Starting recovery of all containers Sep 08 10:06:56.286567 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:56.285586 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-137.ec2.internal.18d35086a826927d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-137.ec2.internal,UID:ip-10-0-138-137.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-137.ec2.internal,},FirstTimestamp:2026-09-08 10:06:56.273158781 +0000 UTC m=+0.317262858,LastTimestamp:2026-09-08 10:06:56.273158781 +0000 UTC m=+0.317262858,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-137.ec2.internal,}" Sep 08 10:06:56.287005 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:56.286751 2568 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-137.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 08 10:06:56.297741 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.297725 2568 manager.go:354] Recovery completed Sep 08 10:06:56.299771 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.299690 2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 10:06:56.301818 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.301804 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 10:06:56.303740 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.303722 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-137.ec2.internal" event="NodeHasSufficientMemory" Sep 08 10:06:56.303815 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.303754 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-137.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 10:06:56.303815 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.303769 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-137.ec2.internal" event="NodeHasSufficientPID" Sep 08 10:06:56.304276 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.304263 2568 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 08 10:06:56.304319 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.304277 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 10:06:56.304353 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.304277 2568 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 08 10:06:56.304353 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.304349 2568 state_mem.go:36] "Initialized new in-memory state store" Sep 08 10:06:56.304935 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.304920 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-137.ec2.internal" event="NodeHasSufficientMemory" Sep 08 10:06:56.304978 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.304944 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-137.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 10:06:56.304978 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.304954 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-137.ec2.internal" event="NodeHasSufficientPID" Sep 08 10:06:56.306503 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.306491 2568 policy_none.go:49] "None policy: Start" Sep 08 10:06:56.306542 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.306508 2568 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 08 10:06:56.306542 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.306518 2568 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 08 10:06:56.308421 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.308400 2568 policy_none.go:47] "Start" Sep 08 10:06:56.351143 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.351127 2568 manager.go:337] "Starting Device Plugin manager" Sep 08 10:06:56.373814 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:56.351163 2568 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 08 10:06:56.373814 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.351173 2568 server.go:87] "Starting device plugin registration server" Sep 08 10:06:56.373814 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.351457 2568 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 08 10:06:56.373814 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.351467 2568 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 08 10:06:56.373814 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.351559 2568 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 08 10:06:56.373814 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.351616 2568 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 08 10:06:56.373814 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.351622 2568 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 08 10:06:56.373814 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:56.352121 2568 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 08 10:06:56.373814 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:56.352174 2568 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-137.ec2.internal\" not found" Sep 08 10:06:56.396895 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.396873 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 08 10:06:56.398126 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.398109 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 08 10:06:56.398194 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.398160 2568 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 08 10:06:56.398194 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.398187 2568 kubelet.go:2448] "Starting kubelet main sync loop" Sep 08 10:06:56.398263 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:56.398232 2568 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 08 10:06:56.403008 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.402991 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 10:06:56.451712 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.451654 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 10:06:56.452422 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.452384 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-137.ec2.internal" event="NodeHasSufficientMemory" Sep 08 10:06:56.452497 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.452437 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-137.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 10:06:56.452497 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.452451 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-137.ec2.internal" event="NodeHasSufficientPID" Sep 08 10:06:56.452497 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.452472 2568 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-137.ec2.internal" Sep 08 10:06:56.460538 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.460525 2568 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-137.ec2.internal" Sep 08 10:06:56.460583 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:56.460544 2568 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-137.ec2.internal\": node \"ip-10-0-138-137.ec2.internal\" not found" Sep 08 10:06:56.480257 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:56.480235 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-137.ec2.internal\" not found" Sep 08 10:06:56.498879 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.498857 2568 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-138-137.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-137.ec2.internal"] Sep 08 10:06:56.498961 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.498910 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 10:06:56.499619 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.499606 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-137.ec2.internal" event="NodeHasSufficientMemory" Sep 08 10:06:56.499673 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.499631 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-137.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 10:06:56.499673 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.499640 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-137.ec2.internal" event="NodeHasSufficientPID" Sep 08 10:06:56.501846 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.501835 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 10:06:56.501976 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.501961 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-137.ec2.internal" Sep 08 10:06:56.502012 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.501990 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 10:06:56.502550 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.502521 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-137.ec2.internal" event="NodeHasSufficientMemory" Sep 08 10:06:56.502550 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.502536 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-137.ec2.internal" event="NodeHasSufficientMemory" Sep 08 10:06:56.502651 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.502555 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-137.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 10:06:56.502651 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.502555 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-137.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 10:06:56.502651 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.502580 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-137.ec2.internal" event="NodeHasSufficientPID" Sep 08 10:06:56.502651 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.502566 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-137.ec2.internal" event="NodeHasSufficientPID" Sep 08 10:06:56.504757 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.504741 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-137.ec2.internal" Sep 08 10:06:56.504847 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.504765 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 10:06:56.505402 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.505388 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-137.ec2.internal" event="NodeHasSufficientMemory" Sep 08 10:06:56.505488 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.505433 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-137.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 10:06:56.505488 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.505449 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-137.ec2.internal" event="NodeHasSufficientPID" Sep 08 10:06:56.542955 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:56.542938 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-137.ec2.internal\" not found" node="ip-10-0-138-137.ec2.internal" Sep 08 10:06:56.547155 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:56.547141 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-137.ec2.internal\" not found" node="ip-10-0-138-137.ec2.internal" Sep 08 10:06:56.581205 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:56.581185 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-137.ec2.internal\" not found" Sep 08 10:06:56.681278 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:56.681259 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-137.ec2.internal\" not found" Sep 08 10:06:56.685532 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.685518 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a0020a1e2b212dc8803848e0c1e1357d-config\") pod \"kube-apiserver-proxy-ip-10-0-138-137.ec2.internal\" (UID: \"a0020a1e2b212dc8803848e0c1e1357d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-137.ec2.internal" Sep 08 10:06:56.685584 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.685540 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5f178c2ff5fdb9f03911e9c42444ad6e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-137.ec2.internal\" (UID: \"5f178c2ff5fdb9f03911e9c42444ad6e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-137.ec2.internal" Sep 08 10:06:56.685584 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.685556 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5f178c2ff5fdb9f03911e9c42444ad6e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-137.ec2.internal\" (UID: \"5f178c2ff5fdb9f03911e9c42444ad6e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-137.ec2.internal" Sep 08 10:06:56.781880 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:56.781836 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-137.ec2.internal\" not found" Sep 08 10:06:56.786093 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.786077 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a0020a1e2b212dc8803848e0c1e1357d-config\") pod \"kube-apiserver-proxy-ip-10-0-138-137.ec2.internal\" (UID: \"a0020a1e2b212dc8803848e0c1e1357d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-137.ec2.internal" Sep 08 10:06:56.786138 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.786100 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5f178c2ff5fdb9f03911e9c42444ad6e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-137.ec2.internal\" (UID: \"5f178c2ff5fdb9f03911e9c42444ad6e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-137.ec2.internal" Sep 08 10:06:56.786138 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.786118 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5f178c2ff5fdb9f03911e9c42444ad6e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-137.ec2.internal\" (UID: \"5f178c2ff5fdb9f03911e9c42444ad6e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-137.ec2.internal" Sep 08 10:06:56.786198 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.786141 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5f178c2ff5fdb9f03911e9c42444ad6e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-137.ec2.internal\" (UID: \"5f178c2ff5fdb9f03911e9c42444ad6e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-137.ec2.internal" Sep 08 10:06:56.786198 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.786169 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5f178c2ff5fdb9f03911e9c42444ad6e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-137.ec2.internal\" (UID: \"5f178c2ff5fdb9f03911e9c42444ad6e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-137.ec2.internal" Sep 08 10:06:56.786198 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.786173 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a0020a1e2b212dc8803848e0c1e1357d-config\") pod \"kube-apiserver-proxy-ip-10-0-138-137.ec2.internal\" (UID: \"a0020a1e2b212dc8803848e0c1e1357d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-137.ec2.internal" Sep 08 10:06:56.846248 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.846231 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-137.ec2.internal" Sep 08 10:06:56.852210 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:56.852197 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-137.ec2.internal" Sep 08 10:06:56.882356 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:56.882333 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-137.ec2.internal\" not found" Sep 08 10:06:56.982920 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:56.982893 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-137.ec2.internal\" not found" Sep 08 10:06:57.083399 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:57.083356 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-137.ec2.internal\" not found" Sep 08 10:06:57.101642 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.101625 2568 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 10:06:57.184191 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.184172 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-137.ec2.internal" Sep 08 10:06:57.193887 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.193873 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 08 10:06:57.195510 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.195499 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-137.ec2.internal" Sep 08 10:06:57.204796 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.204783 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 08 10:06:57.221936 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.221922 2568 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 08 10:06:57.222051 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.222033 2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 08 10:06:57.222087 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.222041 2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 08 10:06:57.222087 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.222041 2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 08 10:06:57.263231 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.263210 2568 apiserver.go:52] "Watching apiserver" Sep 08 10:06:57.280795 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.280775 2568 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 08 10:06:57.281149 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.281129 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-rx2jz","openshift-network-operator/iptables-alerter-7xfcp","openshift-ovn-kubernetes/ovnkube-node-k296b","kube-system/global-pull-secret-syncer-pv8mz","kube-system/kube-apiserver-proxy-ip-10-0-138-137.ec2.internal","openshift-cluster-node-tuning-operator/tuned-ntxft","openshift-dns/node-resolver-vngnd","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-137.ec2.internal","openshift-multus/multus-additional-cni-plugins-w5t97","openshift-multus/multus-p9x7d","openshift-multus/network-metrics-daemon-mbs64","kube-system/konnectivity-agent-x4v9d","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djz4q","openshift-image-registry/node-ca-4gqq4"] Sep 08 10:06:57.283546 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.283533 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 08 10:06:57.283591 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.283556 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vngnd" Sep 08 10:06:57.284617 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.284572 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-07 10:01:56 +0000 UTC" deadline="2028-02-21 20:19:49.490801715 +0000 UTC" Sep 08 10:06:57.284705 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.284616 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12754h12m52.206188901s" Sep 08 10:06:57.285730 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.285713 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 08 10:06:57.285862 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.285850 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 08 10:06:57.285928 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.285915 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbs64" Sep 08 10:06:57.285975 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.285965 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-nbxgx\"" Sep 08 10:06:57.286025 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:57.285987 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mbs64" podUID="3b4e4397-1898-4201-9dcd-b5775ee1789c" Sep 08 10:06:57.288244 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.288225 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.288600 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.288573 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b4e4397-1898-4201-9dcd-b5775ee1789c-metrics-certs\") pod \"network-metrics-daemon-mbs64\" (UID: \"3b4e4397-1898-4201-9dcd-b5775ee1789c\") " pod="openshift-multus/network-metrics-daemon-mbs64" Sep 08 10:06:57.288679 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.288612 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gmqj\" (UniqueName: \"kubernetes.io/projected/3b4e4397-1898-4201-9dcd-b5775ee1789c-kube-api-access-2gmqj\") pod \"network-metrics-daemon-mbs64\" (UID: \"3b4e4397-1898-4201-9dcd-b5775ee1789c\") " pod="openshift-multus/network-metrics-daemon-mbs64" Sep 08 10:06:57.288679 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.288637 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/29e04309-88ba-49b2-aab0-4b68da8efd96-hosts-file\") pod \"node-resolver-vngnd\" (UID: \"29e04309-88ba-49b2-aab0-4b68da8efd96\") " pod="openshift-dns/node-resolver-vngnd" Sep 08 10:06:57.288679 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.288661 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29e04309-88ba-49b2-aab0-4b68da8efd96-tmp-dir\") pod \"node-resolver-vngnd\" (UID: \"29e04309-88ba-49b2-aab0-4b68da8efd96\") " pod="openshift-dns/node-resolver-vngnd" Sep 08 10:06:57.288787 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.288683 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tzlp\" (UniqueName: \"kubernetes.io/projected/29e04309-88ba-49b2-aab0-4b68da8efd96-kube-api-access-6tzlp\") pod \"node-resolver-vngnd\" (UID: \"29e04309-88ba-49b2-aab0-4b68da8efd96\") " pod="openshift-dns/node-resolver-vngnd" Sep 08 10:06:57.290364 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.290350 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 08 10:06:57.290461 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.290368 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-d76qv\"" Sep 08 10:06:57.290461 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.290449 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv8mz" Sep 08 10:06:57.290568 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:57.290516 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pv8mz" podUID="b608607f-8020-4d4e-9f43-b369b983a3b6" Sep 08 10:06:57.290661 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.290646 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 08 10:06:57.290711 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.290693 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 08 10:06:57.290711 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.290706 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 08 10:06:57.290834 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.290751 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 08 10:06:57.290934 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.290920 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 08 10:06:57.292663 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.292651 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4gqq4" Sep 08 10:06:57.294718 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.294703 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-pxj5m\"" Sep 08 10:06:57.294788 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.294762 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 08 10:06:57.294988 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.294974 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.295063 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.295046 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 08 10:06:57.295121 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.294979 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 08 10:06:57.297110 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.297092 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 08 10:06:57.297180 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.297090 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 08 10:06:57.297278 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.297264 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-w5t97" Sep 08 10:06:57.297543 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.297524 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 08 10:06:57.297709 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.297696 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-dz8jq\"" Sep 08 10:06:57.299493 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.299476 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djz4q" Sep 08 10:06:57.299571 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.299554 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 08 10:06:57.299629 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.299569 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 08 10:06:57.299629 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.299577 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 08 10:06:57.299904 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.299891 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 08 10:06:57.300550 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.300063 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 08 10:06:57.300550 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.300207 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-wbk6l\"" Sep 08 10:06:57.301786 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.301768 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 08 10:06:57.301861 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.301818 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7xfcp" Sep 08 10:06:57.302157 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.302140 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 08 10:06:57.302240 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.302186 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 08 10:06:57.302240 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.302208 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-88rln\"" Sep 08 10:06:57.304019 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.304001 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 08 10:06:57.304096 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.304029 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 08 10:06:57.304145 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.304099 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xngbl\"" Sep 08 10:06:57.304440 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.304423 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 08 10:06:57.307028 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.305167 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.307749 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.307731 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx2jz" Sep 08 10:06:57.307839 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:57.307814 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rx2jz" podUID="808f3bbd-c348-4c64-8ea7-e2e692594df8" Sep 08 10:06:57.308458 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.308436 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 08 10:06:57.308552 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.308443 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-6b4qm\"" Sep 08 10:06:57.310598 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.310581 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-x4v9d" Sep 08 10:06:57.312870 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.312846 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 08 10:06:57.312956 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.312849 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 08 10:06:57.313011 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.312957 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-c6228\"" Sep 08 10:06:57.321760 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.321744 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-4mt92" Sep 08 10:06:57.330118 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.330103 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-4mt92" Sep 08 10:06:57.362922 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.362909 2568 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 10:06:57.385875 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.385860 2568 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 08 10:06:57.388818 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.388794 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-etc-tuned\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.388907 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.388832 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-cnibin\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.388907 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.388862 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-host-var-lib-cni-bin\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.388907 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.388886 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-etc-kubernetes\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.389058 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.388922 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-run-ovn\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.389058 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.388952 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8ee82827-bbff-454b-b538-e76bf6fcf14d-env-overrides\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.389058 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.388976 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-etc-kubernetes\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.389058 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.389003 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9a781adb-4df3-440c-965a-1aa5e72f7271-tuning-conf-dir\") pod \"multus-additional-cni-plugins-w5t97\" (UID: \"9a781adb-4df3-440c-965a-1aa5e72f7271\") " pod="openshift-multus/multus-additional-cni-plugins-w5t97" Sep 08 10:06:57.389058 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.389048 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tzlp\" (UniqueName: \"kubernetes.io/projected/29e04309-88ba-49b2-aab0-4b68da8efd96-kube-api-access-6tzlp\") pod \"node-resolver-vngnd\" (UID: \"29e04309-88ba-49b2-aab0-4b68da8efd96\") " pod="openshift-dns/node-resolver-vngnd" Sep 08 10:06:57.389282 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.389076 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-host-run-k8s-cni-cncf-io\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.389282 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.389120 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-var-lib-openvswitch\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.389282 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.389210 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-run-openvswitch\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.389282 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.389237 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8ee82827-bbff-454b-b538-e76bf6fcf14d-ovnkube-config\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.389282 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.389259 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-etc-sysconfig\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.389467 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.389284 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9a781adb-4df3-440c-965a-1aa5e72f7271-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-w5t97\" (UID: \"9a781adb-4df3-440c-965a-1aa5e72f7271\") " pod="openshift-multus/multus-additional-cni-plugins-w5t97" Sep 08 10:06:57.389467 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.389320 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a6ea5044-04d6-44ae-a82e-35a4422fef08-konnectivity-ca\") pod \"konnectivity-agent-x4v9d\" (UID: \"a6ea5044-04d6-44ae-a82e-35a4422fef08\") " pod="kube-system/konnectivity-agent-x4v9d" Sep 08 10:06:57.389467 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.389362 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b608607f-8020-4d4e-9f43-b369b983a3b6-kubelet-config\") pod \"global-pull-secret-syncer-pv8mz\" (UID: \"b608607f-8020-4d4e-9f43-b369b983a3b6\") " pod="kube-system/global-pull-secret-syncer-pv8mz" Sep 08 10:06:57.389467 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.389394 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cfac6ed7-3115-4f50-9271-6a240f950ae5-host\") pod \"node-ca-4gqq4\" (UID: \"cfac6ed7-3115-4f50-9271-6a240f950ae5\") " pod="openshift-image-registry/node-ca-4gqq4" Sep 08 10:06:57.389467 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.389442 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-etc-sysctl-d\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.389467 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.389463 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-sys\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.389636 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.389477 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-multus-cni-dir\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.389636 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.389491 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1e3e9550-3196-4649-89d7-01df1adc42a1-socket-dir\") pod \"aws-ebs-csi-driver-node-djz4q\" (UID: \"1e3e9550-3196-4649-89d7-01df1adc42a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djz4q" Sep 08 10:06:57.389636 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.389529 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2ea3bd6d-f1ab-4952-9091-c32bb4870a89-iptables-alerter-script\") pod \"iptables-alerter-7xfcp\" (UID: \"2ea3bd6d-f1ab-4952-9091-c32bb4870a89\") " pod="openshift-network-operator/iptables-alerter-7xfcp" Sep 08 10:06:57.389636 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.389556 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6c5qw\" (UniqueName: \"kubernetes.io/projected/2ea3bd6d-f1ab-4952-9091-c32bb4870a89-kube-api-access-6c5qw\") pod \"iptables-alerter-7xfcp\" (UID: \"2ea3bd6d-f1ab-4952-9091-c32bb4870a89\") " pod="openshift-network-operator/iptables-alerter-7xfcp" Sep 08 10:06:57.389636 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.389574 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-host-var-lib-kubelet\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.389636 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.389614 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-host\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.389795 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.389642 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-multus-daemon-config\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.389795 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.389682 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtccv\" (UniqueName: \"kubernetes.io/projected/8ee82827-bbff-454b-b538-e76bf6fcf14d-kube-api-access-rtccv\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.389795 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.389702 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-var-lib-kubelet\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.389795 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.389721 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b4e4397-1898-4201-9dcd-b5775ee1789c-metrics-certs\") pod \"network-metrics-daemon-mbs64\" (UID: \"3b4e4397-1898-4201-9dcd-b5775ee1789c\") " pod="openshift-multus/network-metrics-daemon-mbs64" Sep 08 10:06:57.389795 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.389753 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/29e04309-88ba-49b2-aab0-4b68da8efd96-hosts-file\") pod \"node-resolver-vngnd\" (UID: \"29e04309-88ba-49b2-aab0-4b68da8efd96\") " pod="openshift-dns/node-resolver-vngnd" Sep 08 10:06:57.389930 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:57.389801 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 10:06:57.389930 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:57.389852 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b4e4397-1898-4201-9dcd-b5775ee1789c-metrics-certs podName:3b4e4397-1898-4201-9dcd-b5775ee1789c nodeName:}" failed. No retries permitted until 2026-09-08 10:06:57.889837691 +0000 UTC m=+1.933941737 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b4e4397-1898-4201-9dcd-b5775ee1789c-metrics-certs") pod "network-metrics-daemon-mbs64" (UID: "3b4e4397-1898-4201-9dcd-b5775ee1789c") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 10:06:57.389930 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.389867 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a6ea5044-04d6-44ae-a82e-35a4422fef08-agent-certs\") pod \"konnectivity-agent-x4v9d\" (UID: \"a6ea5044-04d6-44ae-a82e-35a4422fef08\") " pod="kube-system/konnectivity-agent-x4v9d" Sep 08 10:06:57.389930 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.389886 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/29e04309-88ba-49b2-aab0-4b68da8efd96-hosts-file\") pod \"node-resolver-vngnd\" (UID: \"29e04309-88ba-49b2-aab0-4b68da8efd96\") " pod="openshift-dns/node-resolver-vngnd" Sep 08 10:06:57.389930 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.389912 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-host-slash\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.389930 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.389926 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bq9tj\" (UniqueName: \"kubernetes.io/projected/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-kube-api-access-bq9tj\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.390110 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.389966 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-host-run-netns\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.390110 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.389993 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-etc-openvswitch\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.390110 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390015 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.390110 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390036 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-cni-binary-copy\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.390110 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390052 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b608607f-8020-4d4e-9f43-b369b983a3b6-dbus\") pod \"global-pull-secret-syncer-pv8mz\" (UID: \"b608607f-8020-4d4e-9f43-b369b983a3b6\") " pod="kube-system/global-pull-secret-syncer-pv8mz" Sep 08 10:06:57.390110 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390065 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-host-var-lib-cni-multus\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.390302 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390091 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjrcr\" (UniqueName: \"kubernetes.io/projected/808f3bbd-c348-4c64-8ea7-e2e692594df8-kube-api-access-rjrcr\") pod \"network-check-target-rx2jz\" (UID: \"808f3bbd-c348-4c64-8ea7-e2e692594df8\") " pod="openshift-network-diagnostics/network-check-target-rx2jz" Sep 08 10:06:57.390302 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390127 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1e3e9550-3196-4649-89d7-01df1adc42a1-device-dir\") pod \"aws-ebs-csi-driver-node-djz4q\" (UID: \"1e3e9550-3196-4649-89d7-01df1adc42a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djz4q" Sep 08 10:06:57.390302 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390148 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-system-cni-dir\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.390302 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390172 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-hostroot\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.390302 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390195 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-systemd-units\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.390302 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390218 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8ee82827-bbff-454b-b538-e76bf6fcf14d-ovn-node-metrics-cert\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.390302 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390233 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-etc-sysctl-conf\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.390302 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390289 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-tmp\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.390691 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390307 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njgqp\" (UniqueName: \"kubernetes.io/projected/9a781adb-4df3-440c-965a-1aa5e72f7271-kube-api-access-njgqp\") pod \"multus-additional-cni-plugins-w5t97\" (UID: \"9a781adb-4df3-440c-965a-1aa5e72f7271\") " pod="openshift-multus/multus-additional-cni-plugins-w5t97" Sep 08 10:06:57.390691 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390321 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-multus-conf-dir\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.390691 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390334 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-log-socket\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.390691 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390348 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cfac6ed7-3115-4f50-9271-6a240f950ae5-serviceca\") pod \"node-ca-4gqq4\" (UID: \"cfac6ed7-3115-4f50-9271-6a240f950ae5\") " pod="openshift-image-registry/node-ca-4gqq4" Sep 08 10:06:57.390691 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390392 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9a781adb-4df3-440c-965a-1aa5e72f7271-cni-binary-copy\") pod \"multus-additional-cni-plugins-w5t97\" (UID: \"9a781adb-4df3-440c-965a-1aa5e72f7271\") " pod="openshift-multus/multus-additional-cni-plugins-w5t97" Sep 08 10:06:57.390691 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390448 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1e3e9550-3196-4649-89d7-01df1adc42a1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-djz4q\" (UID: \"1e3e9550-3196-4649-89d7-01df1adc42a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djz4q" Sep 08 10:06:57.390691 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390473 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2ea3bd6d-f1ab-4952-9091-c32bb4870a89-host-slash\") pod \"iptables-alerter-7xfcp\" (UID: \"2ea3bd6d-f1ab-4952-9091-c32bb4870a89\") " pod="openshift-network-operator/iptables-alerter-7xfcp" Sep 08 10:06:57.390691 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390496 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1e3e9550-3196-4649-89d7-01df1adc42a1-sys-fs\") pod \"aws-ebs-csi-driver-node-djz4q\" (UID: \"1e3e9550-3196-4649-89d7-01df1adc42a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djz4q" Sep 08 10:06:57.390691 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390519 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnmws\" (UniqueName: \"kubernetes.io/projected/1e3e9550-3196-4649-89d7-01df1adc42a1-kube-api-access-xnmws\") pod \"aws-ebs-csi-driver-node-djz4q\" (UID: \"1e3e9550-3196-4649-89d7-01df1adc42a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djz4q" Sep 08 10:06:57.390691 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390542 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-host-run-netns\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.390691 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390563 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm2tb\" (UniqueName: \"kubernetes.io/projected/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-kube-api-access-nm2tb\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.390691 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390602 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxvj8\" (UniqueName: \"kubernetes.io/projected/cfac6ed7-3115-4f50-9271-6a240f950ae5-kube-api-access-dxvj8\") pod \"node-ca-4gqq4\" (UID: \"cfac6ed7-3115-4f50-9271-6a240f950ae5\") " pod="openshift-image-registry/node-ca-4gqq4" Sep 08 10:06:57.390691 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390626 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9a781adb-4df3-440c-965a-1aa5e72f7271-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-w5t97\" (UID: \"9a781adb-4df3-440c-965a-1aa5e72f7271\") " pod="openshift-multus/multus-additional-cni-plugins-w5t97" Sep 08 10:06:57.390691 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390641 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-host-cni-bin\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.390691 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390660 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-lib-modules\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.390691 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390677 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-multus-socket-dir-parent\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.391445 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390691 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8ee82827-bbff-454b-b538-e76bf6fcf14d-ovnkube-script-lib\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.391445 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390704 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-etc-modprobe-d\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.391445 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390716 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-run\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.391445 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390730 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-run-systemd\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.391445 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390749 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-node-log\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.391445 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390763 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-host-cni-netd\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.391445 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390792 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9a781adb-4df3-440c-965a-1aa5e72f7271-cnibin\") pod \"multus-additional-cni-plugins-w5t97\" (UID: \"9a781adb-4df3-440c-965a-1aa5e72f7271\") " pod="openshift-multus/multus-additional-cni-plugins-w5t97" Sep 08 10:06:57.391445 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390817 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-host-run-ovn-kubernetes\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.391445 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390862 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29e04309-88ba-49b2-aab0-4b68da8efd96-tmp-dir\") pod \"node-resolver-vngnd\" (UID: \"29e04309-88ba-49b2-aab0-4b68da8efd96\") " pod="openshift-dns/node-resolver-vngnd" Sep 08 10:06:57.391445 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390893 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1e3e9550-3196-4649-89d7-01df1adc42a1-registration-dir\") pod \"aws-ebs-csi-driver-node-djz4q\" (UID: \"1e3e9550-3196-4649-89d7-01df1adc42a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djz4q" Sep 08 10:06:57.391445 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390917 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b608607f-8020-4d4e-9f43-b369b983a3b6-original-pull-secret\") pod \"global-pull-secret-syncer-pv8mz\" (UID: \"b608607f-8020-4d4e-9f43-b369b983a3b6\") " pod="kube-system/global-pull-secret-syncer-pv8mz" Sep 08 10:06:57.391445 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390935 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-host-run-multus-certs\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.391445 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390954 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-host-kubelet\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.391445 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390970 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a781adb-4df3-440c-965a-1aa5e72f7271-system-cni-dir\") pod \"multus-additional-cni-plugins-w5t97\" (UID: \"9a781adb-4df3-440c-965a-1aa5e72f7271\") " pod="openshift-multus/multus-additional-cni-plugins-w5t97" Sep 08 10:06:57.391445 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.390995 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gmqj\" (UniqueName: \"kubernetes.io/projected/3b4e4397-1898-4201-9dcd-b5775ee1789c-kube-api-access-2gmqj\") pod \"network-metrics-daemon-mbs64\" (UID: \"3b4e4397-1898-4201-9dcd-b5775ee1789c\") " pod="openshift-multus/network-metrics-daemon-mbs64" Sep 08 10:06:57.391445 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.391038 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1e3e9550-3196-4649-89d7-01df1adc42a1-etc-selinux\") pod \"aws-ebs-csi-driver-node-djz4q\" (UID: \"1e3e9550-3196-4649-89d7-01df1adc42a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djz4q" Sep 08 10:06:57.392079 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.391084 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-etc-systemd\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.392079 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.391146 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29e04309-88ba-49b2-aab0-4b68da8efd96-tmp-dir\") pod \"node-resolver-vngnd\" (UID: \"29e04309-88ba-49b2-aab0-4b68da8efd96\") " pod="openshift-dns/node-resolver-vngnd" Sep 08 10:06:57.395151 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.395134 2568 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 08 10:06:57.398644 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.398624 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tzlp\" (UniqueName: \"kubernetes.io/projected/29e04309-88ba-49b2-aab0-4b68da8efd96-kube-api-access-6tzlp\") pod \"node-resolver-vngnd\" (UID: \"29e04309-88ba-49b2-aab0-4b68da8efd96\") " pod="openshift-dns/node-resolver-vngnd" Sep 08 10:06:57.399792 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.399773 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gmqj\" (UniqueName: \"kubernetes.io/projected/3b4e4397-1898-4201-9dcd-b5775ee1789c-kube-api-access-2gmqj\") pod \"network-metrics-daemon-mbs64\" (UID: \"3b4e4397-1898-4201-9dcd-b5775ee1789c\") " pod="openshift-multus/network-metrics-daemon-mbs64" Sep 08 10:06:57.491694 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.491674 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-host-run-ovn-kubernetes\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.491817 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.491701 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1e3e9550-3196-4649-89d7-01df1adc42a1-registration-dir\") pod \"aws-ebs-csi-driver-node-djz4q\" (UID: \"1e3e9550-3196-4649-89d7-01df1adc42a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djz4q" Sep 08 10:06:57.491817 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.491715 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b608607f-8020-4d4e-9f43-b369b983a3b6-original-pull-secret\") pod \"global-pull-secret-syncer-pv8mz\" (UID: \"b608607f-8020-4d4e-9f43-b369b983a3b6\") " pod="kube-system/global-pull-secret-syncer-pv8mz" Sep 08 10:06:57.491817 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.491729 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-host-run-multus-certs\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.491817 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.491742 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-host-kubelet\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.491817 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.491723 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-host-run-ovn-kubernetes\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.491817 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.491787 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-host-kubelet\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.491817 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.491788 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1e3e9550-3196-4649-89d7-01df1adc42a1-registration-dir\") pod \"aws-ebs-csi-driver-node-djz4q\" (UID: \"1e3e9550-3196-4649-89d7-01df1adc42a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djz4q" Sep 08 10:06:57.491817 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.491804 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-host-run-multus-certs\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.492110 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:57.491827 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 10:06:57.492110 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.491865 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a781adb-4df3-440c-965a-1aa5e72f7271-system-cni-dir\") pod \"multus-additional-cni-plugins-w5t97\" (UID: \"9a781adb-4df3-440c-965a-1aa5e72f7271\") " pod="openshift-multus/multus-additional-cni-plugins-w5t97" Sep 08 10:06:57.492110 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:57.491891 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b608607f-8020-4d4e-9f43-b369b983a3b6-original-pull-secret podName:b608607f-8020-4d4e-9f43-b369b983a3b6 nodeName:}" failed. No retries permitted until 2026-09-08 10:06:57.991867332 +0000 UTC m=+2.035971394 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b608607f-8020-4d4e-9f43-b369b983a3b6-original-pull-secret") pod "global-pull-secret-syncer-pv8mz" (UID: "b608607f-8020-4d4e-9f43-b369b983a3b6") : object "kube-system"/"original-pull-secret" not registered Sep 08 10:06:57.492110 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.491908 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a781adb-4df3-440c-965a-1aa5e72f7271-system-cni-dir\") pod \"multus-additional-cni-plugins-w5t97\" (UID: \"9a781adb-4df3-440c-965a-1aa5e72f7271\") " pod="openshift-multus/multus-additional-cni-plugins-w5t97" Sep 08 10:06:57.492110 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.491914 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1e3e9550-3196-4649-89d7-01df1adc42a1-etc-selinux\") pod \"aws-ebs-csi-driver-node-djz4q\" (UID: \"1e3e9550-3196-4649-89d7-01df1adc42a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djz4q" Sep 08 10:06:57.492110 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.491937 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-etc-systemd\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.492110 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.491971 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-etc-tuned\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.492110 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.491991 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-cnibin\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.492110 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492004 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-host-var-lib-cni-bin\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.492110 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492023 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-etc-kubernetes\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.492110 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492046 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-run-ovn\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.492110 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492055 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-etc-systemd\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.492110 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492069 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8ee82827-bbff-454b-b538-e76bf6fcf14d-env-overrides\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.492110 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492062 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1e3e9550-3196-4649-89d7-01df1adc42a1-etc-selinux\") pod \"aws-ebs-csi-driver-node-djz4q\" (UID: \"1e3e9550-3196-4649-89d7-01df1adc42a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djz4q" Sep 08 10:06:57.492110 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492111 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-etc-kubernetes\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.492837 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492129 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-cnibin\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.492837 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492137 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9a781adb-4df3-440c-965a-1aa5e72f7271-tuning-conf-dir\") pod \"multus-additional-cni-plugins-w5t97\" (UID: \"9a781adb-4df3-440c-965a-1aa5e72f7271\") " pod="openshift-multus/multus-additional-cni-plugins-w5t97" Sep 08 10:06:57.492837 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492141 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-etc-kubernetes\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.492837 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492198 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-host-var-lib-cni-bin\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.492837 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492199 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-run-ovn\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.492837 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492222 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-host-run-k8s-cni-cncf-io\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.492837 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492235 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-etc-kubernetes\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.492837 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492252 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-var-lib-openvswitch\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.492837 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492279 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-host-run-k8s-cni-cncf-io\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.492837 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492278 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-run-openvswitch\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.492837 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492310 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-run-openvswitch\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.492837 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492325 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8ee82827-bbff-454b-b538-e76bf6fcf14d-ovnkube-config\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.492837 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492349 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-var-lib-openvswitch\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.492837 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492351 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9a781adb-4df3-440c-965a-1aa5e72f7271-tuning-conf-dir\") pod \"multus-additional-cni-plugins-w5t97\" (UID: \"9a781adb-4df3-440c-965a-1aa5e72f7271\") " pod="openshift-multus/multus-additional-cni-plugins-w5t97" Sep 08 10:06:57.492837 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492351 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-etc-sysconfig\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.492837 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492386 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9a781adb-4df3-440c-965a-1aa5e72f7271-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-w5t97\" (UID: \"9a781adb-4df3-440c-965a-1aa5e72f7271\") " pod="openshift-multus/multus-additional-cni-plugins-w5t97" Sep 08 10:06:57.492837 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492393 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-etc-sysconfig\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.493682 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492430 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a6ea5044-04d6-44ae-a82e-35a4422fef08-konnectivity-ca\") pod \"konnectivity-agent-x4v9d\" (UID: \"a6ea5044-04d6-44ae-a82e-35a4422fef08\") " pod="kube-system/konnectivity-agent-x4v9d" Sep 08 10:06:57.493682 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492452 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b608607f-8020-4d4e-9f43-b369b983a3b6-kubelet-config\") pod \"global-pull-secret-syncer-pv8mz\" (UID: \"b608607f-8020-4d4e-9f43-b369b983a3b6\") " pod="kube-system/global-pull-secret-syncer-pv8mz" Sep 08 10:06:57.493682 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492473 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cfac6ed7-3115-4f50-9271-6a240f950ae5-host\") pod \"node-ca-4gqq4\" (UID: \"cfac6ed7-3115-4f50-9271-6a240f950ae5\") " pod="openshift-image-registry/node-ca-4gqq4" Sep 08 10:06:57.493682 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492493 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-etc-sysctl-d\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.493682 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492512 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-sys\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.493682 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492534 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-multus-cni-dir\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.493682 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492559 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1e3e9550-3196-4649-89d7-01df1adc42a1-socket-dir\") pod \"aws-ebs-csi-driver-node-djz4q\" (UID: \"1e3e9550-3196-4649-89d7-01df1adc42a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djz4q" Sep 08 10:06:57.493682 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492581 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2ea3bd6d-f1ab-4952-9091-c32bb4870a89-iptables-alerter-script\") pod \"iptables-alerter-7xfcp\" (UID: \"2ea3bd6d-f1ab-4952-9091-c32bb4870a89\") " pod="openshift-network-operator/iptables-alerter-7xfcp" Sep 08 10:06:57.493682 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492582 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b608607f-8020-4d4e-9f43-b369b983a3b6-kubelet-config\") pod \"global-pull-secret-syncer-pv8mz\" (UID: \"b608607f-8020-4d4e-9f43-b369b983a3b6\") " pod="kube-system/global-pull-secret-syncer-pv8mz" Sep 08 10:06:57.493682 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492605 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6c5qw\" (UniqueName: \"kubernetes.io/projected/2ea3bd6d-f1ab-4952-9091-c32bb4870a89-kube-api-access-6c5qw\") pod \"iptables-alerter-7xfcp\" (UID: \"2ea3bd6d-f1ab-4952-9091-c32bb4870a89\") " pod="openshift-network-operator/iptables-alerter-7xfcp" Sep 08 10:06:57.493682 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492628 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8ee82827-bbff-454b-b538-e76bf6fcf14d-env-overrides\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.493682 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492630 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cfac6ed7-3115-4f50-9271-6a240f950ae5-host\") pod \"node-ca-4gqq4\" (UID: \"cfac6ed7-3115-4f50-9271-6a240f950ae5\") " pod="openshift-image-registry/node-ca-4gqq4" Sep 08 10:06:57.493682 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492705 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-multus-cni-dir\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.493682 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492711 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-sys\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.493682 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492715 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-host-var-lib-kubelet\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.493682 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492820 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-etc-sysctl-d\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.493682 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492826 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-host\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.493682 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492867 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-multus-daemon-config\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.494580 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492877 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-host\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.494580 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492894 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtccv\" (UniqueName: \"kubernetes.io/projected/8ee82827-bbff-454b-b538-e76bf6fcf14d-kube-api-access-rtccv\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.494580 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492911 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8ee82827-bbff-454b-b538-e76bf6fcf14d-ovnkube-config\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.494580 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492920 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-var-lib-kubelet\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.494580 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492992 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a6ea5044-04d6-44ae-a82e-35a4422fef08-agent-certs\") pod \"konnectivity-agent-x4v9d\" (UID: \"a6ea5044-04d6-44ae-a82e-35a4422fef08\") " pod="kube-system/konnectivity-agent-x4v9d" Sep 08 10:06:57.494580 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.492992 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9a781adb-4df3-440c-965a-1aa5e72f7271-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-w5t97\" (UID: \"9a781adb-4df3-440c-965a-1aa5e72f7271\") " pod="openshift-multus/multus-additional-cni-plugins-w5t97" Sep 08 10:06:57.494580 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.493176 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-host-var-lib-kubelet\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.494580 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.493216 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-host-slash\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.494580 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.493217 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1e3e9550-3196-4649-89d7-01df1adc42a1-socket-dir\") pod \"aws-ebs-csi-driver-node-djz4q\" (UID: \"1e3e9550-3196-4649-89d7-01df1adc42a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djz4q" Sep 08 10:06:57.494580 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.493257 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a6ea5044-04d6-44ae-a82e-35a4422fef08-konnectivity-ca\") pod \"konnectivity-agent-x4v9d\" (UID: \"a6ea5044-04d6-44ae-a82e-35a4422fef08\") " pod="kube-system/konnectivity-agent-x4v9d" Sep 08 10:06:57.494580 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.493264 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-var-lib-kubelet\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.494580 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.493273 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2ea3bd6d-f1ab-4952-9091-c32bb4870a89-iptables-alerter-script\") pod \"iptables-alerter-7xfcp\" (UID: \"2ea3bd6d-f1ab-4952-9091-c32bb4870a89\") " pod="openshift-network-operator/iptables-alerter-7xfcp" Sep 08 10:06:57.494580 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.493304 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq9tj\" (UniqueName: \"kubernetes.io/projected/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-kube-api-access-bq9tj\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.494580 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.493307 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-host-slash\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.494580 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.493440 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-host-run-netns\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.494580 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.493470 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-etc-openvswitch\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.494580 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.493495 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.495481 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.493519 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-cni-binary-copy\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.495481 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.493544 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b608607f-8020-4d4e-9f43-b369b983a3b6-dbus\") pod \"global-pull-secret-syncer-pv8mz\" (UID: \"b608607f-8020-4d4e-9f43-b369b983a3b6\") " pod="kube-system/global-pull-secret-syncer-pv8mz" Sep 08 10:06:57.495481 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.493561 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-etc-openvswitch\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.495481 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.493608 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-host-run-netns\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.495481 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.493612 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b608607f-8020-4d4e-9f43-b369b983a3b6-dbus\") pod \"global-pull-secret-syncer-pv8mz\" (UID: \"b608607f-8020-4d4e-9f43-b369b983a3b6\") " pod="kube-system/global-pull-secret-syncer-pv8mz" Sep 08 10:06:57.495481 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.493634 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-host-var-lib-cni-multus\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.495481 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.493658 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.495481 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.493670 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjrcr\" (UniqueName: \"kubernetes.io/projected/808f3bbd-c348-4c64-8ea7-e2e692594df8-kube-api-access-rjrcr\") pod \"network-check-target-rx2jz\" (UID: \"808f3bbd-c348-4c64-8ea7-e2e692594df8\") " pod="openshift-network-diagnostics/network-check-target-rx2jz" Sep 08 10:06:57.495481 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.493697 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-host-var-lib-cni-multus\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.495481 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.493725 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1e3e9550-3196-4649-89d7-01df1adc42a1-device-dir\") pod \"aws-ebs-csi-driver-node-djz4q\" (UID: \"1e3e9550-3196-4649-89d7-01df1adc42a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djz4q" Sep 08 10:06:57.495481 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.493749 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-system-cni-dir\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.495481 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.493776 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-hostroot\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.495481 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.493798 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-multus-daemon-config\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.495481 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.493805 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1e3e9550-3196-4649-89d7-01df1adc42a1-device-dir\") pod \"aws-ebs-csi-driver-node-djz4q\" (UID: \"1e3e9550-3196-4649-89d7-01df1adc42a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djz4q" Sep 08 10:06:57.495481 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.493800 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-systemd-units\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.495481 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.493852 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8ee82827-bbff-454b-b538-e76bf6fcf14d-ovn-node-metrics-cert\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.495481 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.493894 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-etc-sysctl-conf\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.496131 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.493911 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-system-cni-dir\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.496131 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.493887 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-hostroot\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.496131 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.493925 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-systemd-units\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.496131 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.493917 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-tmp\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.496131 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.493989 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-cni-binary-copy\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.496131 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.493995 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njgqp\" (UniqueName: \"kubernetes.io/projected/9a781adb-4df3-440c-965a-1aa5e72f7271-kube-api-access-njgqp\") pod \"multus-additional-cni-plugins-w5t97\" (UID: \"9a781adb-4df3-440c-965a-1aa5e72f7271\") " pod="openshift-multus/multus-additional-cni-plugins-w5t97" Sep 08 10:06:57.496131 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.494038 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-etc-sysctl-conf\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.496131 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.494061 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-multus-conf-dir\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.496131 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.494085 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-log-socket\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.496131 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.494101 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cfac6ed7-3115-4f50-9271-6a240f950ae5-serviceca\") pod \"node-ca-4gqq4\" (UID: \"cfac6ed7-3115-4f50-9271-6a240f950ae5\") " pod="openshift-image-registry/node-ca-4gqq4" Sep 08 10:06:57.496131 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.494114 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9a781adb-4df3-440c-965a-1aa5e72f7271-cni-binary-copy\") pod \"multus-additional-cni-plugins-w5t97\" (UID: \"9a781adb-4df3-440c-965a-1aa5e72f7271\") " pod="openshift-multus/multus-additional-cni-plugins-w5t97" Sep 08 10:06:57.496131 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.494128 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1e3e9550-3196-4649-89d7-01df1adc42a1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-djz4q\" (UID: \"1e3e9550-3196-4649-89d7-01df1adc42a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djz4q" Sep 08 10:06:57.496131 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.494154 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2ea3bd6d-f1ab-4952-9091-c32bb4870a89-host-slash\") pod \"iptables-alerter-7xfcp\" (UID: \"2ea3bd6d-f1ab-4952-9091-c32bb4870a89\") " pod="openshift-network-operator/iptables-alerter-7xfcp" Sep 08 10:06:57.496131 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.494168 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-log-socket\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.496131 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.494179 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1e3e9550-3196-4649-89d7-01df1adc42a1-sys-fs\") pod \"aws-ebs-csi-driver-node-djz4q\" (UID: \"1e3e9550-3196-4649-89d7-01df1adc42a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djz4q" Sep 08 10:06:57.496131 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.494206 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-multus-conf-dir\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.496131 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.494206 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnmws\" (UniqueName: \"kubernetes.io/projected/1e3e9550-3196-4649-89d7-01df1adc42a1-kube-api-access-xnmws\") pod \"aws-ebs-csi-driver-node-djz4q\" (UID: \"1e3e9550-3196-4649-89d7-01df1adc42a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djz4q" Sep 08 10:06:57.496131 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.494250 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-host-run-netns\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.496729 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.494273 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm2tb\" (UniqueName: \"kubernetes.io/projected/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-kube-api-access-nm2tb\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.496729 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.494287 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1e3e9550-3196-4649-89d7-01df1adc42a1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-djz4q\" (UID: \"1e3e9550-3196-4649-89d7-01df1adc42a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djz4q" Sep 08 10:06:57.496729 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.494295 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxvj8\" (UniqueName: \"kubernetes.io/projected/cfac6ed7-3115-4f50-9271-6a240f950ae5-kube-api-access-dxvj8\") pod \"node-ca-4gqq4\" (UID: \"cfac6ed7-3115-4f50-9271-6a240f950ae5\") " pod="openshift-image-registry/node-ca-4gqq4" Sep 08 10:06:57.496729 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.494330 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9a781adb-4df3-440c-965a-1aa5e72f7271-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-w5t97\" (UID: \"9a781adb-4df3-440c-965a-1aa5e72f7271\") " pod="openshift-multus/multus-additional-cni-plugins-w5t97" Sep 08 10:06:57.496729 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.494360 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-host-cni-bin\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.496729 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.494383 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-lib-modules\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.496729 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.494429 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-multus-socket-dir-parent\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.496729 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.494455 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8ee82827-bbff-454b-b538-e76bf6fcf14d-ovnkube-script-lib\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.496729 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.494479 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-etc-modprobe-d\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.496729 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.494500 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-run\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.496729 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.494542 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1e3e9550-3196-4649-89d7-01df1adc42a1-sys-fs\") pod \"aws-ebs-csi-driver-node-djz4q\" (UID: \"1e3e9550-3196-4649-89d7-01df1adc42a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djz4q" Sep 08 10:06:57.496729 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.494541 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-run-systemd\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.496729 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.494582 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-run-systemd\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.496729 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.494591 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-node-log\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.496729 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.494629 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-node-log\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.496729 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.494850 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2ea3bd6d-f1ab-4952-9091-c32bb4870a89-host-slash\") pod \"iptables-alerter-7xfcp\" (UID: \"2ea3bd6d-f1ab-4952-9091-c32bb4870a89\") " pod="openshift-network-operator/iptables-alerter-7xfcp" Sep 08 10:06:57.496729 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.494885 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-host-run-netns\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.497404 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.494745 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9a781adb-4df3-440c-965a-1aa5e72f7271-cni-binary-copy\") pod \"multus-additional-cni-plugins-w5t97\" (UID: \"9a781adb-4df3-440c-965a-1aa5e72f7271\") " pod="openshift-multus/multus-additional-cni-plugins-w5t97" Sep 08 10:06:57.497404 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.495031 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-multus-socket-dir-parent\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.497404 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.495037 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9a781adb-4df3-440c-965a-1aa5e72f7271-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-w5t97\" (UID: \"9a781adb-4df3-440c-965a-1aa5e72f7271\") " pod="openshift-multus/multus-additional-cni-plugins-w5t97" Sep 08 10:06:57.497404 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.495080 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-host-cni-netd\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.497404 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.495106 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9a781adb-4df3-440c-965a-1aa5e72f7271-cnibin\") pod \"multus-additional-cni-plugins-w5t97\" (UID: \"9a781adb-4df3-440c-965a-1aa5e72f7271\") " pod="openshift-multus/multus-additional-cni-plugins-w5t97" Sep 08 10:06:57.497404 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.495153 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-lib-modules\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.497404 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.495150 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-etc-tuned\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.497404 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.495168 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-etc-modprobe-d\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.497404 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.495081 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-host-cni-bin\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.497404 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.495171 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9a781adb-4df3-440c-965a-1aa5e72f7271-cnibin\") pod \"multus-additional-cni-plugins-w5t97\" (UID: \"9a781adb-4df3-440c-965a-1aa5e72f7271\") " pod="openshift-multus/multus-additional-cni-plugins-w5t97" Sep 08 10:06:57.497404 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.495211 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8ee82827-bbff-454b-b538-e76bf6fcf14d-host-cni-netd\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.497404 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.495251 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-run\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.497404 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.495299 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cfac6ed7-3115-4f50-9271-6a240f950ae5-serviceca\") pod \"node-ca-4gqq4\" (UID: \"cfac6ed7-3115-4f50-9271-6a240f950ae5\") " pod="openshift-image-registry/node-ca-4gqq4" Sep 08 10:06:57.497404 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.495652 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8ee82827-bbff-454b-b538-e76bf6fcf14d-ovnkube-script-lib\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.497404 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.495677 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a6ea5044-04d6-44ae-a82e-35a4422fef08-agent-certs\") pod \"konnectivity-agent-x4v9d\" (UID: \"a6ea5044-04d6-44ae-a82e-35a4422fef08\") " pod="kube-system/konnectivity-agent-x4v9d" Sep 08 10:06:57.497404 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.496720 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-tmp\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.498043 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.497502 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8ee82827-bbff-454b-b538-e76bf6fcf14d-ovn-node-metrics-cert\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.500108 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:57.500089 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 10:06:57.500200 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:57.500112 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 10:06:57.500200 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:57.500125 2568 projected.go:196] Error preparing data for projected volume kube-api-access-rjrcr for pod openshift-network-diagnostics/network-check-target-rx2jz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 10:06:57.500200 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:57.500194 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/808f3bbd-c348-4c64-8ea7-e2e692594df8-kube-api-access-rjrcr podName:808f3bbd-c348-4c64-8ea7-e2e692594df8 nodeName:}" failed. No retries permitted until 2026-09-08 10:06:58.000177527 +0000 UTC m=+2.044281592 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rjrcr" (UniqueName: "kubernetes.io/projected/808f3bbd-c348-4c64-8ea7-e2e692594df8-kube-api-access-rjrcr") pod "network-check-target-rx2jz" (UID: "808f3bbd-c348-4c64-8ea7-e2e692594df8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 10:06:57.502397 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.502373 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6c5qw\" (UniqueName: \"kubernetes.io/projected/2ea3bd6d-f1ab-4952-9091-c32bb4870a89-kube-api-access-6c5qw\") pod \"iptables-alerter-7xfcp\" (UID: \"2ea3bd6d-f1ab-4952-9091-c32bb4870a89\") " pod="openshift-network-operator/iptables-alerter-7xfcp" Sep 08 10:06:57.502611 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.502589 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxvj8\" (UniqueName: \"kubernetes.io/projected/cfac6ed7-3115-4f50-9271-6a240f950ae5-kube-api-access-dxvj8\") pod \"node-ca-4gqq4\" (UID: \"cfac6ed7-3115-4f50-9271-6a240f950ae5\") " pod="openshift-image-registry/node-ca-4gqq4" Sep 08 10:06:57.502704 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.502684 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bq9tj\" (UniqueName: \"kubernetes.io/projected/ba8a6738-a445-4fdc-ad9e-e326d4ad1626-kube-api-access-bq9tj\") pod \"tuned-ntxft\" (UID: \"ba8a6738-a445-4fdc-ad9e-e326d4ad1626\") " pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.502936 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.502911 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm2tb\" (UniqueName: \"kubernetes.io/projected/a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395-kube-api-access-nm2tb\") pod \"multus-p9x7d\" (UID: \"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395\") " pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.503078 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.503060 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njgqp\" (UniqueName: \"kubernetes.io/projected/9a781adb-4df3-440c-965a-1aa5e72f7271-kube-api-access-njgqp\") pod \"multus-additional-cni-plugins-w5t97\" (UID: \"9a781adb-4df3-440c-965a-1aa5e72f7271\") " pod="openshift-multus/multus-additional-cni-plugins-w5t97" Sep 08 10:06:57.503464 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.503442 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtccv\" (UniqueName: \"kubernetes.io/projected/8ee82827-bbff-454b-b538-e76bf6fcf14d-kube-api-access-rtccv\") pod \"ovnkube-node-k296b\" (UID: \"8ee82827-bbff-454b-b538-e76bf6fcf14d\") " pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.503793 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.503778 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnmws\" (UniqueName: \"kubernetes.io/projected/1e3e9550-3196-4649-89d7-01df1adc42a1-kube-api-access-xnmws\") pod \"aws-ebs-csi-driver-node-djz4q\" (UID: \"1e3e9550-3196-4649-89d7-01df1adc42a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djz4q" Sep 08 10:06:57.603281 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.603227 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vngnd" Sep 08 10:06:57.610931 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.610913 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:06:57.622645 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:57.622457 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29e04309_88ba_49b2_aab0_4b68da8efd96.slice/crio-344f65105105f0e90e3c22f94eaedeb9100ef0db6179cc5a59d07d0948989edf WatchSource:0}: Error finding container 344f65105105f0e90e3c22f94eaedeb9100ef0db6179cc5a59d07d0948989edf: Status 404 returned error can't find the container with id 344f65105105f0e90e3c22f94eaedeb9100ef0db6179cc5a59d07d0948989edf Sep 08 10:06:57.622896 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:57.622875 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ee82827_bbff_454b_b538_e76bf6fcf14d.slice/crio-79e04a640c34afb960480e187069dab8e954ca810bccd782563e558ef9da610b WatchSource:0}: Error finding container 79e04a640c34afb960480e187069dab8e954ca810bccd782563e558ef9da610b: Status 404 returned error can't find the container with id 79e04a640c34afb960480e187069dab8e954ca810bccd782563e558ef9da610b Sep 08 10:06:57.625002 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:57.624983 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0020a1e2b212dc8803848e0c1e1357d.slice/crio-3cc6bd452c6a310b41adde058d202777bb2811a19556594c82b3a318994d8ac0 WatchSource:0}: Error finding container 3cc6bd452c6a310b41adde058d202777bb2811a19556594c82b3a318994d8ac0: Status 404 returned error can't find the container with id 3cc6bd452c6a310b41adde058d202777bb2811a19556594c82b3a318994d8ac0 Sep 08 10:06:57.629365 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.629351 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 08 10:06:57.633054 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.633038 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4gqq4" Sep 08 10:06:57.642369 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:57.642348 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfac6ed7_3115_4f50_9271_6a240f950ae5.slice/crio-082449d42f18613c9a25ce1921e87cb805d603f65a2e78d8a5b7b03ce80564a7 WatchSource:0}: Error finding container 082449d42f18613c9a25ce1921e87cb805d603f65a2e78d8a5b7b03ce80564a7: Status 404 returned error can't find the container with id 082449d42f18613c9a25ce1921e87cb805d603f65a2e78d8a5b7b03ce80564a7 Sep 08 10:06:57.648396 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.648381 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ntxft" Sep 08 10:06:57.655706 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:57.655686 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba8a6738_a445_4fdc_ad9e_e326d4ad1626.slice/crio-267cf3c0bd9ebcfafac99fadb1536e048c889f43db93e16ac74f2285f29f0f5a WatchSource:0}: Error finding container 267cf3c0bd9ebcfafac99fadb1536e048c889f43db93e16ac74f2285f29f0f5a: Status 404 returned error can't find the container with id 267cf3c0bd9ebcfafac99fadb1536e048c889f43db93e16ac74f2285f29f0f5a Sep 08 10:06:57.663102 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.663085 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-w5t97" Sep 08 10:06:57.670441 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:57.670403 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a781adb_4df3_440c_965a_1aa5e72f7271.slice/crio-e7b716a6011addce9d3eb25e2e47ecbed76e3606da877ce7fe42e5071fbf1ce7 WatchSource:0}: Error finding container e7b716a6011addce9d3eb25e2e47ecbed76e3606da877ce7fe42e5071fbf1ce7: Status 404 returned error can't find the container with id e7b716a6011addce9d3eb25e2e47ecbed76e3606da877ce7fe42e5071fbf1ce7 Sep 08 10:06:57.678648 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.678629 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djz4q" Sep 08 10:06:57.685732 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:57.685716 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e3e9550_3196_4649_89d7_01df1adc42a1.slice/crio-f51ceea53bef883f29f63797e0efe221068d7288c5e8b194b5ba0a03d77fc1b4 WatchSource:0}: Error finding container f51ceea53bef883f29f63797e0efe221068d7288c5e8b194b5ba0a03d77fc1b4: Status 404 returned error can't find the container with id f51ceea53bef883f29f63797e0efe221068d7288c5e8b194b5ba0a03d77fc1b4 Sep 08 10:06:57.691155 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.691142 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7xfcp" Sep 08 10:06:57.698213 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:57.698198 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ea3bd6d_f1ab_4952_9091_c32bb4870a89.slice/crio-22bb7c0a93bfa635f775ce10c669453949e527058347c05a44710bbbeb13fcd4 WatchSource:0}: Error finding container 22bb7c0a93bfa635f775ce10c669453949e527058347c05a44710bbbeb13fcd4: Status 404 returned error can't find the container with id 22bb7c0a93bfa635f775ce10c669453949e527058347c05a44710bbbeb13fcd4 Sep 08 10:06:57.715389 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.715370 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-p9x7d" Sep 08 10:06:57.719991 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.719976 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-x4v9d" Sep 08 10:06:57.724193 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:57.724177 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3abc8d2_5c6c_45f3_a8e9_b79d30b6b395.slice/crio-28321897e82b1d4759fb1fd1a9f3cf04872a9ee3d3c95db456399e6c4e0c4e5e WatchSource:0}: Error finding container 28321897e82b1d4759fb1fd1a9f3cf04872a9ee3d3c95db456399e6c4e0c4e5e: Status 404 returned error can't find the container with id 28321897e82b1d4759fb1fd1a9f3cf04872a9ee3d3c95db456399e6c4e0c4e5e Sep 08 10:06:57.728500 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:06:57.728479 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6ea5044_04d6_44ae_a82e_35a4422fef08.slice/crio-ee9df822d7a8fa63422b3b0c4a4f37ae57f82f6a4f128a1369428c8584853465 WatchSource:0}: Error finding container ee9df822d7a8fa63422b3b0c4a4f37ae57f82f6a4f128a1369428c8584853465: Status 404 returned error can't find the container with id ee9df822d7a8fa63422b3b0c4a4f37ae57f82f6a4f128a1369428c8584853465 Sep 08 10:06:57.899608 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:57.899589 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b4e4397-1898-4201-9dcd-b5775ee1789c-metrics-certs\") pod \"network-metrics-daemon-mbs64\" (UID: \"3b4e4397-1898-4201-9dcd-b5775ee1789c\") " pod="openshift-multus/network-metrics-daemon-mbs64" Sep 08 10:06:57.899702 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:57.899689 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 10:06:57.899744 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:57.899738 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b4e4397-1898-4201-9dcd-b5775ee1789c-metrics-certs podName:3b4e4397-1898-4201-9dcd-b5775ee1789c nodeName:}" failed. No retries permitted until 2026-09-08 10:06:58.899724034 +0000 UTC m=+2.943828083 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b4e4397-1898-4201-9dcd-b5775ee1789c-metrics-certs") pod "network-metrics-daemon-mbs64" (UID: "3b4e4397-1898-4201-9dcd-b5775ee1789c") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 10:06:58.000947 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:58.000922 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b608607f-8020-4d4e-9f43-b369b983a3b6-original-pull-secret\") pod \"global-pull-secret-syncer-pv8mz\" (UID: \"b608607f-8020-4d4e-9f43-b369b983a3b6\") " pod="kube-system/global-pull-secret-syncer-pv8mz" Sep 08 10:06:58.001022 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:58.000979 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjrcr\" (UniqueName: \"kubernetes.io/projected/808f3bbd-c348-4c64-8ea7-e2e692594df8-kube-api-access-rjrcr\") pod \"network-check-target-rx2jz\" (UID: \"808f3bbd-c348-4c64-8ea7-e2e692594df8\") " pod="openshift-network-diagnostics/network-check-target-rx2jz" Sep 08 10:06:58.001081 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:58.001069 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 10:06:58.001118 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:58.001102 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 10:06:58.001118 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:58.001115 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 10:06:58.001181 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:58.001120 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b608607f-8020-4d4e-9f43-b369b983a3b6-original-pull-secret podName:b608607f-8020-4d4e-9f43-b369b983a3b6 nodeName:}" failed. No retries permitted until 2026-09-08 10:06:59.001106043 +0000 UTC m=+3.045210112 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b608607f-8020-4d4e-9f43-b369b983a3b6-original-pull-secret") pod "global-pull-secret-syncer-pv8mz" (UID: "b608607f-8020-4d4e-9f43-b369b983a3b6") : object "kube-system"/"original-pull-secret" not registered Sep 08 10:06:58.001181 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:58.001126 2568 projected.go:196] Error preparing data for projected volume kube-api-access-rjrcr for pod openshift-network-diagnostics/network-check-target-rx2jz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 10:06:58.001254 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:58.001181 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/808f3bbd-c348-4c64-8ea7-e2e692594df8-kube-api-access-rjrcr podName:808f3bbd-c348-4c64-8ea7-e2e692594df8 nodeName:}" failed. No retries permitted until 2026-09-08 10:06:59.001169879 +0000 UTC m=+3.045273939 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rjrcr" (UniqueName: "kubernetes.io/projected/808f3bbd-c348-4c64-8ea7-e2e692594df8-kube-api-access-rjrcr") pod "network-check-target-rx2jz" (UID: "808f3bbd-c348-4c64-8ea7-e2e692594df8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 10:06:58.268758 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:58.268505 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 10:06:58.332692 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:58.332650 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 10:01:57 +0000 UTC" deadline="2028-04-25 17:28:24.801238603 +0000 UTC" Sep 08 10:06:58.332692 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:58.332690 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14287h21m26.468553003s" Sep 08 10:06:58.414538 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:58.414484 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-x4v9d" event={"ID":"a6ea5044-04d6-44ae-a82e-35a4422fef08","Type":"ContainerStarted","Data":"ee9df822d7a8fa63422b3b0c4a4f37ae57f82f6a4f128a1369428c8584853465"} Sep 08 10:06:58.416672 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:58.416646 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-p9x7d" event={"ID":"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395","Type":"ContainerStarted","Data":"28321897e82b1d4759fb1fd1a9f3cf04872a9ee3d3c95db456399e6c4e0c4e5e"} Sep 08 10:06:58.418064 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:58.418034 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7xfcp" event={"ID":"2ea3bd6d-f1ab-4952-9091-c32bb4870a89","Type":"ContainerStarted","Data":"22bb7c0a93bfa635f775ce10c669453949e527058347c05a44710bbbeb13fcd4"} Sep 08 10:06:58.419433 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:58.419402 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djz4q" event={"ID":"1e3e9550-3196-4649-89d7-01df1adc42a1","Type":"ContainerStarted","Data":"f51ceea53bef883f29f63797e0efe221068d7288c5e8b194b5ba0a03d77fc1b4"} Sep 08 10:06:58.421040 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:58.421013 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w5t97" event={"ID":"9a781adb-4df3-440c-965a-1aa5e72f7271","Type":"ContainerStarted","Data":"e7b716a6011addce9d3eb25e2e47ecbed76e3606da877ce7fe42e5071fbf1ce7"} Sep 08 10:06:58.422724 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:58.422693 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vngnd" event={"ID":"29e04309-88ba-49b2-aab0-4b68da8efd96","Type":"ContainerStarted","Data":"344f65105105f0e90e3c22f94eaedeb9100ef0db6179cc5a59d07d0948989edf"} Sep 08 10:06:58.424264 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:58.424239 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-137.ec2.internal" event={"ID":"a0020a1e2b212dc8803848e0c1e1357d","Type":"ContainerStarted","Data":"3cc6bd452c6a310b41adde058d202777bb2811a19556594c82b3a318994d8ac0"} Sep 08 10:06:58.425727 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:58.425702 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ntxft" event={"ID":"ba8a6738-a445-4fdc-ad9e-e326d4ad1626","Type":"ContainerStarted","Data":"267cf3c0bd9ebcfafac99fadb1536e048c889f43db93e16ac74f2285f29f0f5a"} Sep 08 10:06:58.427340 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:58.427318 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4gqq4" event={"ID":"cfac6ed7-3115-4f50-9271-6a240f950ae5","Type":"ContainerStarted","Data":"082449d42f18613c9a25ce1921e87cb805d603f65a2e78d8a5b7b03ce80564a7"} Sep 08 10:06:58.429072 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:58.429049 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k296b" event={"ID":"8ee82827-bbff-454b-b538-e76bf6fcf14d","Type":"ContainerStarted","Data":"79e04a640c34afb960480e187069dab8e954ca810bccd782563e558ef9da610b"} Sep 08 10:06:58.819080 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:58.819049 2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 10:06:58.917654 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:58.917616 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b4e4397-1898-4201-9dcd-b5775ee1789c-metrics-certs\") pod \"network-metrics-daemon-mbs64\" (UID: \"3b4e4397-1898-4201-9dcd-b5775ee1789c\") " pod="openshift-multus/network-metrics-daemon-mbs64" Sep 08 10:06:58.917854 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:58.917832 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 10:06:58.918010 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:58.917994 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b4e4397-1898-4201-9dcd-b5775ee1789c-metrics-certs podName:3b4e4397-1898-4201-9dcd-b5775ee1789c nodeName:}" failed. No retries permitted until 2026-09-08 10:07:00.917969123 +0000 UTC m=+4.962073234 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b4e4397-1898-4201-9dcd-b5775ee1789c-metrics-certs") pod "network-metrics-daemon-mbs64" (UID: "3b4e4397-1898-4201-9dcd-b5775ee1789c") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 10:06:59.018664 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:59.018632 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b608607f-8020-4d4e-9f43-b369b983a3b6-original-pull-secret\") pod \"global-pull-secret-syncer-pv8mz\" (UID: \"b608607f-8020-4d4e-9f43-b369b983a3b6\") " pod="kube-system/global-pull-secret-syncer-pv8mz" Sep 08 10:06:59.018847 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:59.018723 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjrcr\" (UniqueName: \"kubernetes.io/projected/808f3bbd-c348-4c64-8ea7-e2e692594df8-kube-api-access-rjrcr\") pod \"network-check-target-rx2jz\" (UID: \"808f3bbd-c348-4c64-8ea7-e2e692594df8\") " pod="openshift-network-diagnostics/network-check-target-rx2jz" Sep 08 10:06:59.018847 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:59.018804 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 10:06:59.018847 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:59.018829 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 10:06:59.018847 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:59.018846 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 10:06:59.018991 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:59.018858 2568 projected.go:196] Error preparing data for projected volume kube-api-access-rjrcr for pod openshift-network-diagnostics/network-check-target-rx2jz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 10:06:59.018991 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:59.018882 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b608607f-8020-4d4e-9f43-b369b983a3b6-original-pull-secret podName:b608607f-8020-4d4e-9f43-b369b983a3b6 nodeName:}" failed. No retries permitted until 2026-09-08 10:07:01.018864026 +0000 UTC m=+5.062968095 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b608607f-8020-4d4e-9f43-b369b983a3b6-original-pull-secret") pod "global-pull-secret-syncer-pv8mz" (UID: "b608607f-8020-4d4e-9f43-b369b983a3b6") : object "kube-system"/"original-pull-secret" not registered Sep 08 10:06:59.018991 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:59.018899 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/808f3bbd-c348-4c64-8ea7-e2e692594df8-kube-api-access-rjrcr podName:808f3bbd-c348-4c64-8ea7-e2e692594df8 nodeName:}" failed. No retries permitted until 2026-09-08 10:07:01.018888497 +0000 UTC m=+5.062992565 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rjrcr" (UniqueName: "kubernetes.io/projected/808f3bbd-c348-4c64-8ea7-e2e692594df8-kube-api-access-rjrcr") pod "network-check-target-rx2jz" (UID: "808f3bbd-c348-4c64-8ea7-e2e692594df8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 10:06:59.167934 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:59.167857 2568 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 10:06:59.333776 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:59.333711 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 10:01:57 +0000 UTC" deadline="2028-05-22 17:02:37.7079651 +0000 UTC" Sep 08 10:06:59.333776 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:59.333762 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14934h55m38.374207236s" Sep 08 10:06:59.398471 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:59.398392 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx2jz" Sep 08 10:06:59.398471 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:59.398410 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbs64" Sep 08 10:06:59.398471 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:06:59.398410 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv8mz" Sep 08 10:06:59.398599 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:59.398556 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mbs64" podUID="3b4e4397-1898-4201-9dcd-b5775ee1789c" Sep 08 10:06:59.398729 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:59.398700 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pv8mz" podUID="b608607f-8020-4d4e-9f43-b369b983a3b6" Sep 08 10:06:59.398863 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:06:59.398789 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rx2jz" podUID="808f3bbd-c348-4c64-8ea7-e2e692594df8" Sep 08 10:07:00.441116 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:00.441032 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-137.ec2.internal" event={"ID":"a0020a1e2b212dc8803848e0c1e1357d","Type":"ContainerStarted","Data":"d308afc84c745a6592caf8621e19705680566612af6adf1cd7a24c330fe0fd10"} Sep 08 10:07:00.456540 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:00.456467 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-137.ec2.internal" podStartSLOduration=3.456447297 podStartE2EDuration="3.456447297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 10:06:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:07:00.456191169 +0000 UTC m=+4.500295242" watchObservedRunningTime="2026-09-08 10:07:00.456447297 +0000 UTC m=+4.500551369" Sep 08 10:07:00.933315 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:00.933232 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b4e4397-1898-4201-9dcd-b5775ee1789c-metrics-certs\") pod \"network-metrics-daemon-mbs64\" (UID: \"3b4e4397-1898-4201-9dcd-b5775ee1789c\") " pod="openshift-multus/network-metrics-daemon-mbs64" Sep 08 10:07:00.933507 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:00.933400 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 10:07:00.933507 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:00.933487 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b4e4397-1898-4201-9dcd-b5775ee1789c-metrics-certs podName:3b4e4397-1898-4201-9dcd-b5775ee1789c nodeName:}" failed. No retries permitted until 2026-09-08 10:07:04.933448768 +0000 UTC m=+8.977552830 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b4e4397-1898-4201-9dcd-b5775ee1789c-metrics-certs") pod "network-metrics-daemon-mbs64" (UID: "3b4e4397-1898-4201-9dcd-b5775ee1789c") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 10:07:01.034455 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:01.033634 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b608607f-8020-4d4e-9f43-b369b983a3b6-original-pull-secret\") pod \"global-pull-secret-syncer-pv8mz\" (UID: \"b608607f-8020-4d4e-9f43-b369b983a3b6\") " pod="kube-system/global-pull-secret-syncer-pv8mz" Sep 08 10:07:01.034455 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:01.033777 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjrcr\" (UniqueName: \"kubernetes.io/projected/808f3bbd-c348-4c64-8ea7-e2e692594df8-kube-api-access-rjrcr\") pod \"network-check-target-rx2jz\" (UID: \"808f3bbd-c348-4c64-8ea7-e2e692594df8\") " pod="openshift-network-diagnostics/network-check-target-rx2jz" Sep 08 10:07:01.034455 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:01.033920 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 10:07:01.034455 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:01.033940 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 10:07:01.034455 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:01.033954 2568 projected.go:196] Error preparing data for projected volume kube-api-access-rjrcr for pod openshift-network-diagnostics/network-check-target-rx2jz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 10:07:01.034455 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:01.034012 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/808f3bbd-c348-4c64-8ea7-e2e692594df8-kube-api-access-rjrcr podName:808f3bbd-c348-4c64-8ea7-e2e692594df8 nodeName:}" failed. No retries permitted until 2026-09-08 10:07:05.033993211 +0000 UTC m=+9.078097279 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rjrcr" (UniqueName: "kubernetes.io/projected/808f3bbd-c348-4c64-8ea7-e2e692594df8-kube-api-access-rjrcr") pod "network-check-target-rx2jz" (UID: "808f3bbd-c348-4c64-8ea7-e2e692594df8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 10:07:01.034455 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:01.034386 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 10:07:01.034455 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:01.034427 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b608607f-8020-4d4e-9f43-b369b983a3b6-original-pull-secret podName:b608607f-8020-4d4e-9f43-b369b983a3b6 nodeName:}" failed. No retries permitted until 2026-09-08 10:07:05.034414393 +0000 UTC m=+9.078518442 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b608607f-8020-4d4e-9f43-b369b983a3b6-original-pull-secret") pod "global-pull-secret-syncer-pv8mz" (UID: "b608607f-8020-4d4e-9f43-b369b983a3b6") : object "kube-system"/"original-pull-secret" not registered Sep 08 10:07:01.399550 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:01.399451 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx2jz" Sep 08 10:07:01.399747 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:01.399592 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rx2jz" podUID="808f3bbd-c348-4c64-8ea7-e2e692594df8" Sep 08 10:07:01.400011 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:01.399989 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbs64" Sep 08 10:07:01.400141 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:01.400103 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mbs64" podUID="3b4e4397-1898-4201-9dcd-b5775ee1789c" Sep 08 10:07:01.400223 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:01.400145 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv8mz" Sep 08 10:07:01.400276 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:01.400222 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pv8mz" podUID="b608607f-8020-4d4e-9f43-b369b983a3b6" Sep 08 10:07:03.399021 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:03.398985 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv8mz" Sep 08 10:07:03.399492 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:03.399108 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pv8mz" podUID="b608607f-8020-4d4e-9f43-b369b983a3b6" Sep 08 10:07:03.399568 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:03.399535 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx2jz" Sep 08 10:07:03.399669 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:03.399630 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rx2jz" podUID="808f3bbd-c348-4c64-8ea7-e2e692594df8" Sep 08 10:07:03.399783 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:03.399700 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbs64" Sep 08 10:07:03.399843 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:03.399780 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mbs64" podUID="3b4e4397-1898-4201-9dcd-b5775ee1789c" Sep 08 10:07:04.964618 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:04.964582 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b4e4397-1898-4201-9dcd-b5775ee1789c-metrics-certs\") pod \"network-metrics-daemon-mbs64\" (UID: \"3b4e4397-1898-4201-9dcd-b5775ee1789c\") " pod="openshift-multus/network-metrics-daemon-mbs64" Sep 08 10:07:04.965100 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:04.964727 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 10:07:04.965100 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:04.964782 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b4e4397-1898-4201-9dcd-b5775ee1789c-metrics-certs podName:3b4e4397-1898-4201-9dcd-b5775ee1789c nodeName:}" failed. No retries permitted until 2026-09-08 10:07:12.964764218 +0000 UTC m=+17.008868265 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b4e4397-1898-4201-9dcd-b5775ee1789c-metrics-certs") pod "network-metrics-daemon-mbs64" (UID: "3b4e4397-1898-4201-9dcd-b5775ee1789c") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 10:07:05.065649 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:05.065614 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjrcr\" (UniqueName: \"kubernetes.io/projected/808f3bbd-c348-4c64-8ea7-e2e692594df8-kube-api-access-rjrcr\") pod \"network-check-target-rx2jz\" (UID: \"808f3bbd-c348-4c64-8ea7-e2e692594df8\") " pod="openshift-network-diagnostics/network-check-target-rx2jz" Sep 08 10:07:05.065791 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:05.065680 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b608607f-8020-4d4e-9f43-b369b983a3b6-original-pull-secret\") pod \"global-pull-secret-syncer-pv8mz\" (UID: \"b608607f-8020-4d4e-9f43-b369b983a3b6\") " pod="kube-system/global-pull-secret-syncer-pv8mz" Sep 08 10:07:05.065791 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:05.065761 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 10:07:05.065791 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:05.065784 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 10:07:05.065791 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:05.065791 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 10:07:05.065940 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:05.065805 2568 projected.go:196] Error preparing data for projected volume kube-api-access-rjrcr for pod openshift-network-diagnostics/network-check-target-rx2jz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 10:07:05.065940 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:05.065838 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b608607f-8020-4d4e-9f43-b369b983a3b6-original-pull-secret podName:b608607f-8020-4d4e-9f43-b369b983a3b6 nodeName:}" failed. No retries permitted until 2026-09-08 10:07:13.06582129 +0000 UTC m=+17.109925350 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b608607f-8020-4d4e-9f43-b369b983a3b6-original-pull-secret") pod "global-pull-secret-syncer-pv8mz" (UID: "b608607f-8020-4d4e-9f43-b369b983a3b6") : object "kube-system"/"original-pull-secret" not registered Sep 08 10:07:05.065940 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:05.065862 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/808f3bbd-c348-4c64-8ea7-e2e692594df8-kube-api-access-rjrcr podName:808f3bbd-c348-4c64-8ea7-e2e692594df8 nodeName:}" failed. No retries permitted until 2026-09-08 10:07:13.065845988 +0000 UTC m=+17.109950051 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rjrcr" (UniqueName: "kubernetes.io/projected/808f3bbd-c348-4c64-8ea7-e2e692594df8-kube-api-access-rjrcr") pod "network-check-target-rx2jz" (UID: "808f3bbd-c348-4c64-8ea7-e2e692594df8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 10:07:05.399460 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:05.399370 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx2jz" Sep 08 10:07:05.399613 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:05.399492 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rx2jz" podUID="808f3bbd-c348-4c64-8ea7-e2e692594df8" Sep 08 10:07:05.399920 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:05.399892 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbs64" Sep 08 10:07:05.400023 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:05.400004 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mbs64" podUID="3b4e4397-1898-4201-9dcd-b5775ee1789c" Sep 08 10:07:05.400072 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:05.400047 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv8mz" Sep 08 10:07:05.400132 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:05.400114 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pv8mz" podUID="b608607f-8020-4d4e-9f43-b369b983a3b6" Sep 08 10:07:07.399072 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:07.399035 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv8mz" Sep 08 10:07:07.399675 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:07.399154 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pv8mz" podUID="b608607f-8020-4d4e-9f43-b369b983a3b6" Sep 08 10:07:07.399675 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:07.399559 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx2jz" Sep 08 10:07:07.399675 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:07.399645 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rx2jz" podUID="808f3bbd-c348-4c64-8ea7-e2e692594df8" Sep 08 10:07:07.399823 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:07.399703 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbs64" Sep 08 10:07:07.399823 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:07.399784 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mbs64" podUID="3b4e4397-1898-4201-9dcd-b5775ee1789c" Sep 08 10:07:09.398967 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:09.398933 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbs64" Sep 08 10:07:09.398967 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:09.398966 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx2jz" Sep 08 10:07:09.399585 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:09.399063 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mbs64" podUID="3b4e4397-1898-4201-9dcd-b5775ee1789c" Sep 08 10:07:09.399585 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:09.399145 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rx2jz" podUID="808f3bbd-c348-4c64-8ea7-e2e692594df8" Sep 08 10:07:09.399585 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:09.399192 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv8mz" Sep 08 10:07:09.399585 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:09.399291 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pv8mz" podUID="b608607f-8020-4d4e-9f43-b369b983a3b6" Sep 08 10:07:11.398743 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:11.398712 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv8mz" Sep 08 10:07:11.398743 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:11.398747 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbs64" Sep 08 10:07:11.399234 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:11.398712 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx2jz" Sep 08 10:07:11.399234 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:11.398828 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pv8mz" podUID="b608607f-8020-4d4e-9f43-b369b983a3b6" Sep 08 10:07:11.399234 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:11.398904 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rx2jz" podUID="808f3bbd-c348-4c64-8ea7-e2e692594df8" Sep 08 10:07:11.399234 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:11.398982 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mbs64" podUID="3b4e4397-1898-4201-9dcd-b5775ee1789c" Sep 08 10:07:13.031736 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:13.031696 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b4e4397-1898-4201-9dcd-b5775ee1789c-metrics-certs\") pod \"network-metrics-daemon-mbs64\" (UID: \"3b4e4397-1898-4201-9dcd-b5775ee1789c\") " pod="openshift-multus/network-metrics-daemon-mbs64" Sep 08 10:07:13.032204 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:13.031863 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 10:07:13.032204 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:13.031937 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b4e4397-1898-4201-9dcd-b5775ee1789c-metrics-certs podName:3b4e4397-1898-4201-9dcd-b5775ee1789c nodeName:}" failed. No retries permitted until 2026-09-08 10:07:29.031921879 +0000 UTC m=+33.076025929 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b4e4397-1898-4201-9dcd-b5775ee1789c-metrics-certs") pod "network-metrics-daemon-mbs64" (UID: "3b4e4397-1898-4201-9dcd-b5775ee1789c") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 10:07:13.132718 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:13.132691 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjrcr\" (UniqueName: \"kubernetes.io/projected/808f3bbd-c348-4c64-8ea7-e2e692594df8-kube-api-access-rjrcr\") pod \"network-check-target-rx2jz\" (UID: \"808f3bbd-c348-4c64-8ea7-e2e692594df8\") " pod="openshift-network-diagnostics/network-check-target-rx2jz" Sep 08 10:07:13.132873 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:13.132734 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b608607f-8020-4d4e-9f43-b369b983a3b6-original-pull-secret\") pod \"global-pull-secret-syncer-pv8mz\" (UID: \"b608607f-8020-4d4e-9f43-b369b983a3b6\") " pod="kube-system/global-pull-secret-syncer-pv8mz" Sep 08 10:07:13.132873 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:13.132826 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 10:07:13.132873 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:13.132842 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 10:07:13.132873 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:13.132867 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 10:07:13.132873 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:13.132878 2568 projected.go:196] Error preparing data for projected volume kube-api-access-rjrcr for pod openshift-network-diagnostics/network-check-target-rx2jz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 10:07:13.133101 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:13.132886 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b608607f-8020-4d4e-9f43-b369b983a3b6-original-pull-secret podName:b608607f-8020-4d4e-9f43-b369b983a3b6 nodeName:}" failed. No retries permitted until 2026-09-08 10:07:29.132869873 +0000 UTC m=+33.176973923 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b608607f-8020-4d4e-9f43-b369b983a3b6-original-pull-secret") pod "global-pull-secret-syncer-pv8mz" (UID: "b608607f-8020-4d4e-9f43-b369b983a3b6") : object "kube-system"/"original-pull-secret" not registered Sep 08 10:07:13.133101 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:13.132917 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/808f3bbd-c348-4c64-8ea7-e2e692594df8-kube-api-access-rjrcr podName:808f3bbd-c348-4c64-8ea7-e2e692594df8 nodeName:}" failed. No retries permitted until 2026-09-08 10:07:29.132904536 +0000 UTC m=+33.177008583 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rjrcr" (UniqueName: "kubernetes.io/projected/808f3bbd-c348-4c64-8ea7-e2e692594df8-kube-api-access-rjrcr") pod "network-check-target-rx2jz" (UID: "808f3bbd-c348-4c64-8ea7-e2e692594df8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 10:07:13.398641 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:13.398566 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbs64" Sep 08 10:07:13.398641 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:13.398580 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx2jz" Sep 08 10:07:13.398868 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:13.398690 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mbs64" podUID="3b4e4397-1898-4201-9dcd-b5775ee1789c" Sep 08 10:07:13.398868 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:13.398756 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rx2jz" podUID="808f3bbd-c348-4c64-8ea7-e2e692594df8" Sep 08 10:07:13.398868 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:13.398798 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv8mz" Sep 08 10:07:13.399017 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:13.398875 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pv8mz" podUID="b608607f-8020-4d4e-9f43-b369b983a3b6" Sep 08 10:07:15.398590 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:15.398554 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx2jz" Sep 08 10:07:15.399043 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:15.398668 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbs64" Sep 08 10:07:15.399043 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:15.398674 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rx2jz" podUID="808f3bbd-c348-4c64-8ea7-e2e692594df8" Sep 08 10:07:15.399043 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:15.398705 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv8mz" Sep 08 10:07:15.399043 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:15.398815 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mbs64" podUID="3b4e4397-1898-4201-9dcd-b5775ee1789c" Sep 08 10:07:15.399043 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:15.398911 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pv8mz" podUID="b608607f-8020-4d4e-9f43-b369b983a3b6" Sep 08 10:07:16.470823 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:16.470564 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k296b" event={"ID":"8ee82827-bbff-454b-b538-e76bf6fcf14d","Type":"ContainerStarted","Data":"6b39e4b434e886c0a06ae119e72906703049adeb5b379775f01546006d26ac67"} Sep 08 10:07:16.471180 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:16.470872 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k296b" event={"ID":"8ee82827-bbff-454b-b538-e76bf6fcf14d","Type":"ContainerStarted","Data":"3f3816e14257e72a2e0a3b1c89f27b96f95980b1505455d862b6418e96f9e934"} Sep 08 10:07:16.471604 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:16.471585 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-137.ec2.internal" event={"ID":"5f178c2ff5fdb9f03911e9c42444ad6e","Type":"ContainerStarted","Data":"14d522d32dee428bc251aafd629d3f3a431674c82a799d69f98b3dd7f576e644"} Sep 08 10:07:16.472766 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:16.472746 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-x4v9d" event={"ID":"a6ea5044-04d6-44ae-a82e-35a4422fef08","Type":"ContainerStarted","Data":"59ec87d49152894bb06a703aece131b2b8977d18acaf5adebe9c512941d39527"} Sep 08 10:07:16.474149 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:16.474124 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-p9x7d" event={"ID":"a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395","Type":"ContainerStarted","Data":"6cf3b2d6ddfdf7d6c913ae32f2d3646f2125310f2e9297b21d0543422f22fd60"} Sep 08 10:07:16.475466 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:16.475445 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djz4q" event={"ID":"1e3e9550-3196-4649-89d7-01df1adc42a1","Type":"ContainerStarted","Data":"d8cc307ae6390dc5372b4462bddfc600320a73c4b2d5eaf0046184913364d8b1"} Sep 08 10:07:16.476962 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:16.476941 2568 generic.go:356] "Generic (PLEG): container finished" podID="9a781adb-4df3-440c-965a-1aa5e72f7271" containerID="0269b84c6cadf25bb3c40d722480b80db44181c3f1c0c9730190151f7a90560e" exitCode=0 Sep 08 10:07:16.477027 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:16.477004 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w5t97" event={"ID":"9a781adb-4df3-440c-965a-1aa5e72f7271","Type":"ContainerDied","Data":"0269b84c6cadf25bb3c40d722480b80db44181c3f1c0c9730190151f7a90560e"} Sep 08 10:07:16.478513 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:16.478494 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vngnd" event={"ID":"29e04309-88ba-49b2-aab0-4b68da8efd96","Type":"ContainerStarted","Data":"b18a6d1e2f0c9c25c018bacd3bfedfdc8bfbdbe9441732aa1223382ca1882985"} Sep 08 10:07:16.479854 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:16.479835 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ntxft" event={"ID":"ba8a6738-a445-4fdc-ad9e-e326d4ad1626","Type":"ContainerStarted","Data":"af1dfa38fb28b2a0647258552d90c70bd4538efc1da9aa85ce1723c2c0779312"} Sep 08 10:07:16.481158 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:16.481141 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4gqq4" event={"ID":"cfac6ed7-3115-4f50-9271-6a240f950ae5","Type":"ContainerStarted","Data":"9d83432acee3ffad2c5b25de1f8c35a04ffc68c9e6603c007b5e8e0726a85ff3"} Sep 08 10:07:16.488072 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:16.488037 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-x4v9d" podStartSLOduration=2.493153067 podStartE2EDuration="20.488026064s" totalImagesPullingTime="17.994872997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:06:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:07:16.487550591 +0000 UTC m=+20.531654680" watchObservedRunningTime="2026-09-08 10:07:16.488026064 +0000 UTC m=+20.532130162" Sep 08 10:07:16.503311 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:16.503277 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4gqq4" podStartSLOduration=5.765675324 podStartE2EDuration="20.503266373s" totalImagesPullingTime="14.737591049s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 10:06:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:07:16.503265885 +0000 UTC m=+20.547369965" watchObservedRunningTime="2026-09-08 10:07:16.503266373 +0000 UTC m=+20.547370441" Sep 08 10:07:16.522057 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:16.522026 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ntxft" podStartSLOduration=5.797556322 podStartE2EDuration="20.522015827s" totalImagesPullingTime="14.724459505s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 10:06:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:07:16.521952404 +0000 UTC m=+20.566056497" watchObservedRunningTime="2026-09-08 10:07:16.522015827 +0000 UTC m=+20.566119892" Sep 08 10:07:16.567042 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:16.567003 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-p9x7d" podStartSLOduration=2.5216205240000003 podStartE2EDuration="20.566993332s" totalImagesPullingTime="18.045372808s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 10:06:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:07:16.566811175 +0000 UTC m=+20.610915262" watchObservedRunningTime="2026-09-08 10:07:16.566993332 +0000 UTC m=+20.611097399" Sep 08 10:07:16.583679 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:16.583645 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vngnd" podStartSLOduration=2.488474057 podStartE2EDuration="20.583638346s" totalImagesPullingTime="18.095164289s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 10:06:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:07:16.583007065 +0000 UTC m=+20.627111152" watchObservedRunningTime="2026-09-08 10:07:16.583638346 +0000 UTC m=+20.627742413" Sep 08 10:07:16.967990 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:16.967957 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-x4v9d" Sep 08 10:07:16.968559 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:16.968537 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-x4v9d" Sep 08 10:07:17.399681 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:17.398837 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx2jz" Sep 08 10:07:17.399681 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:17.398952 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rx2jz" podUID="808f3bbd-c348-4c64-8ea7-e2e692594df8" Sep 08 10:07:17.399681 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:17.399370 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbs64" Sep 08 10:07:17.399681 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:17.399505 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mbs64" podUID="3b4e4397-1898-4201-9dcd-b5775ee1789c" Sep 08 10:07:17.399681 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:17.399567 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv8mz" Sep 08 10:07:17.399681 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:17.399643 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pv8mz" podUID="b608607f-8020-4d4e-9f43-b369b983a3b6" Sep 08 10:07:17.484560 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:17.484536 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7xfcp" event={"ID":"2ea3bd6d-f1ab-4952-9091-c32bb4870a89","Type":"ContainerStarted","Data":"a8b0c8b01ebca96b6c0007dec1731e04d73e581a4f4d15587973d64aa2b63f78"} Sep 08 10:07:17.485060 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:17.484713 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-x4v9d" Sep 08 10:07:17.485152 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:17.485135 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-x4v9d" Sep 08 10:07:17.500077 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:17.500034 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-7xfcp" podStartSLOduration=6.818019274 podStartE2EDuration="21.500022286s" totalImagesPullingTime="14.682003012s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 10:06:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:07:17.499604837 +0000 UTC m=+21.543708927" watchObservedRunningTime="2026-09-08 10:07:17.500022286 +0000 UTC m=+21.544126397" Sep 08 10:07:18.490778 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:18.490734 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k296b" event={"ID":"8ee82827-bbff-454b-b538-e76bf6fcf14d","Type":"ContainerStarted","Data":"e32b7bf1a232920e7321943cef2771f20499510db533ef4e05379506f20cd1dd"} Sep 08 10:07:18.491440 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:18.490822 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k296b" event={"ID":"8ee82827-bbff-454b-b538-e76bf6fcf14d","Type":"ContainerStarted","Data":"b3a8f93c683ff9106ea3ca2c70e9161bfd1b44c6138296a329230253471604b2"} Sep 08 10:07:18.491440 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:18.490872 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k296b" event={"ID":"8ee82827-bbff-454b-b538-e76bf6fcf14d","Type":"ContainerStarted","Data":"e70be9a3bb6ff80f5660ccbe3357234f77b37b23b21e11f96fc38fd82dcf2246"} Sep 08 10:07:18.491440 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:18.490914 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k296b" event={"ID":"8ee82827-bbff-454b-b538-e76bf6fcf14d","Type":"ContainerStarted","Data":"ae973f8793125a86792b030c330438da770534678f95ee10cdeece03d64f9680"} Sep 08 10:07:18.493117 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:18.492887 2568 generic.go:356] "Generic (PLEG): container finished" podID="5f178c2ff5fdb9f03911e9c42444ad6e" containerID="7b4ffc89079f3006f93f39761bfc1e445167c0c08709365253725fadae4d24fb" exitCode=0 Sep 08 10:07:18.493866 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:18.493353 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-137.ec2.internal" event={"ID":"5f178c2ff5fdb9f03911e9c42444ad6e","Type":"ContainerDied","Data":"7b4ffc89079f3006f93f39761bfc1e445167c0c08709365253725fadae4d24fb"} Sep 08 10:07:18.902152 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:18.902124 2568 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 08 10:07:19.371939 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:19.371826 2568 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-08T10:07:18.902143528Z","UUID":"22a07f11-979c-46c2-ab4e-4b90917af9cb","Handler":null,"Name":"","Endpoint":""} Sep 08 10:07:19.373799 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:19.373773 2568 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 08 10:07:19.373799 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:19.373805 2568 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 08 10:07:19.399243 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:19.398716 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx2jz" Sep 08 10:07:19.399243 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:19.398733 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbs64" Sep 08 10:07:19.399243 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:19.398733 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv8mz" Sep 08 10:07:19.399243 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:19.398818 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rx2jz" podUID="808f3bbd-c348-4c64-8ea7-e2e692594df8" Sep 08 10:07:19.399243 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:19.398891 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mbs64" podUID="3b4e4397-1898-4201-9dcd-b5775ee1789c" Sep 08 10:07:19.399243 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:19.398948 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pv8mz" podUID="b608607f-8020-4d4e-9f43-b369b983a3b6" Sep 08 10:07:19.497268 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:19.497231 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-137.ec2.internal" event={"ID":"5f178c2ff5fdb9f03911e9c42444ad6e","Type":"ContainerStarted","Data":"1685986a48c50265e035b56944d09094cb218be197815d1d7eae8b4852139382"} Sep 08 10:07:19.499174 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:19.499144 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djz4q" event={"ID":"1e3e9550-3196-4649-89d7-01df1adc42a1","Type":"ContainerStarted","Data":"222e9c5f81813611a961acb3d9a69ee43f62cbfb350ac65b07b1aecef4f0c0ab"} Sep 08 10:07:19.515383 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:19.515344 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-137.ec2.internal" podStartSLOduration=22.515331481 podStartE2EDuration="22.515331481s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 10:06:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:07:19.514406959 +0000 UTC m=+23.558511048" watchObservedRunningTime="2026-09-08 10:07:19.515331481 +0000 UTC m=+23.559435548" Sep 08 10:07:20.504224 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:20.504186 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djz4q" event={"ID":"1e3e9550-3196-4649-89d7-01df1adc42a1","Type":"ContainerStarted","Data":"cf59db3c20a2b0f1e82a555c042aed79928472b7795d03e17487b79b056f6577"} Sep 08 10:07:20.508161 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:20.508136 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k296b" event={"ID":"8ee82827-bbff-454b-b538-e76bf6fcf14d","Type":"ContainerStarted","Data":"1e889df223a7a43eda3a8187ca5b19978f18d1fd8c0641f6bc991bac822c301d"} Sep 08 10:07:20.524159 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:20.523982 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djz4q" podStartSLOduration=3.009553419 podStartE2EDuration="24.523964065s" totalImagesPullingTime="21.514410646s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 10:06:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:07:20.520902619 +0000 UTC m=+24.565006714" watchObservedRunningTime="2026-09-08 10:07:20.523964065 +0000 UTC m=+24.568068113" Sep 08 10:07:21.399026 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:21.398992 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx2jz" Sep 08 10:07:21.399026 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:21.399011 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbs64" Sep 08 10:07:21.399269 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:21.398991 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv8mz" Sep 08 10:07:21.399269 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:21.399139 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rx2jz" podUID="808f3bbd-c348-4c64-8ea7-e2e692594df8" Sep 08 10:07:21.399269 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:21.399225 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pv8mz" podUID="b608607f-8020-4d4e-9f43-b369b983a3b6" Sep 08 10:07:21.399424 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:21.399313 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mbs64" podUID="3b4e4397-1898-4201-9dcd-b5775ee1789c" Sep 08 10:07:22.514323 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:22.514140 2568 generic.go:356] "Generic (PLEG): container finished" podID="9a781adb-4df3-440c-965a-1aa5e72f7271" containerID="23e5395c5ea92fc2f65498f85a59317a0b25277a649f9d48353b3e028f202f1e" exitCode=0 Sep 08 10:07:22.514748 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:22.514220 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w5t97" event={"ID":"9a781adb-4df3-440c-965a-1aa5e72f7271","Type":"ContainerDied","Data":"23e5395c5ea92fc2f65498f85a59317a0b25277a649f9d48353b3e028f202f1e"} Sep 08 10:07:22.518153 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:22.518124 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k296b" event={"ID":"8ee82827-bbff-454b-b538-e76bf6fcf14d","Type":"ContainerStarted","Data":"4ca6fbcbac080bc005746a918a1d30253589689c5ccbb345c2f8e08557e31e62"} Sep 08 10:07:22.518397 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:22.518378 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:07:22.518482 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:22.518469 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:07:22.518553 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:22.518520 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:07:22.534805 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:22.534776 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:07:22.535265 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:22.535250 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:07:23.399132 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:23.399104 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx2jz" Sep 08 10:07:23.399284 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:23.399148 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbs64" Sep 08 10:07:23.399284 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:23.399148 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv8mz" Sep 08 10:07:23.399284 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:23.399219 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rx2jz" podUID="808f3bbd-c348-4c64-8ea7-e2e692594df8" Sep 08 10:07:23.400120 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:23.399458 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mbs64" podUID="3b4e4397-1898-4201-9dcd-b5775ee1789c" Sep 08 10:07:23.400120 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:23.399525 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pv8mz" podUID="b608607f-8020-4d4e-9f43-b369b983a3b6" Sep 08 10:07:23.525608 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:23.525572 2568 generic.go:356] "Generic (PLEG): container finished" podID="9a781adb-4df3-440c-965a-1aa5e72f7271" containerID="160be2ad96dc268603d4428c5587c5f3fbddc1e6697943e3f0cb9d9a0e7b9c9b" exitCode=0 Sep 08 10:07:23.526189 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:23.525646 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w5t97" event={"ID":"9a781adb-4df3-440c-965a-1aa5e72f7271","Type":"ContainerDied","Data":"160be2ad96dc268603d4428c5587c5f3fbddc1e6697943e3f0cb9d9a0e7b9c9b"} Sep 08 10:07:23.552096 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:23.552055 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-k296b" podStartSLOduration=7.860132196 podStartE2EDuration="27.552039514s" totalImagesPullingTime="19.691907318s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 10:06:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:07:22.57303777 +0000 UTC m=+26.617141848" watchObservedRunningTime="2026-09-08 10:07:23.552039514 +0000 UTC m=+27.596143581" Sep 08 10:07:23.878538 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:23.878311 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rx2jz"] Sep 08 10:07:23.878538 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:23.878433 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx2jz" Sep 08 10:07:23.878747 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:23.878546 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rx2jz" podUID="808f3bbd-c348-4c64-8ea7-e2e692594df8" Sep 08 10:07:23.880324 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:23.880299 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pv8mz"] Sep 08 10:07:23.880472 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:23.880375 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv8mz" Sep 08 10:07:23.880528 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:23.880483 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pv8mz" podUID="b608607f-8020-4d4e-9f43-b369b983a3b6" Sep 08 10:07:23.890461 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:23.890432 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mbs64"] Sep 08 10:07:23.890557 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:23.890512 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbs64" Sep 08 10:07:23.890603 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:23.890577 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mbs64" podUID="3b4e4397-1898-4201-9dcd-b5775ee1789c" Sep 08 10:07:25.399379 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:25.399352 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx2jz" Sep 08 10:07:25.399897 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:25.399467 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbs64" Sep 08 10:07:25.399897 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:25.399481 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rx2jz" podUID="808f3bbd-c348-4c64-8ea7-e2e692594df8" Sep 08 10:07:25.399897 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:25.399543 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mbs64" podUID="3b4e4397-1898-4201-9dcd-b5775ee1789c" Sep 08 10:07:25.399897 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:25.399556 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv8mz" Sep 08 10:07:25.399897 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:25.399643 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pv8mz" podUID="b608607f-8020-4d4e-9f43-b369b983a3b6" Sep 08 10:07:26.535080 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:26.534912 2568 generic.go:356] "Generic (PLEG): container finished" podID="9a781adb-4df3-440c-965a-1aa5e72f7271" containerID="ff34127097eeaba7d84d80f521152a194251d0438d63dc991966095f82c1cd09" exitCode=0 Sep 08 10:07:26.535080 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:26.534991 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w5t97" event={"ID":"9a781adb-4df3-440c-965a-1aa5e72f7271","Type":"ContainerDied","Data":"ff34127097eeaba7d84d80f521152a194251d0438d63dc991966095f82c1cd09"} Sep 08 10:07:27.398500 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:27.398473 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbs64" Sep 08 10:07:27.398676 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:27.398473 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv8mz" Sep 08 10:07:27.398676 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:27.398473 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx2jz" Sep 08 10:07:27.398676 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:27.398658 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pv8mz" podUID="b608607f-8020-4d4e-9f43-b369b983a3b6" Sep 08 10:07:27.398799 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:27.398708 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rx2jz" podUID="808f3bbd-c348-4c64-8ea7-e2e692594df8" Sep 08 10:07:27.398799 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:27.398565 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mbs64" podUID="3b4e4397-1898-4201-9dcd-b5775ee1789c" Sep 08 10:07:29.049645 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:29.049610 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b4e4397-1898-4201-9dcd-b5775ee1789c-metrics-certs\") pod \"network-metrics-daemon-mbs64\" (UID: \"3b4e4397-1898-4201-9dcd-b5775ee1789c\") " pod="openshift-multus/network-metrics-daemon-mbs64" Sep 08 10:07:29.050083 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:29.049728 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 10:07:29.050083 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:29.049789 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b4e4397-1898-4201-9dcd-b5775ee1789c-metrics-certs podName:3b4e4397-1898-4201-9dcd-b5775ee1789c nodeName:}" failed. No retries permitted until 2026-09-08 10:08:01.049769274 +0000 UTC m=+65.093873334 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b4e4397-1898-4201-9dcd-b5775ee1789c-metrics-certs") pod "network-metrics-daemon-mbs64" (UID: "3b4e4397-1898-4201-9dcd-b5775ee1789c") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 10:07:29.150915 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:29.150874 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b608607f-8020-4d4e-9f43-b369b983a3b6-original-pull-secret\") pod \"global-pull-secret-syncer-pv8mz\" (UID: \"b608607f-8020-4d4e-9f43-b369b983a3b6\") " pod="kube-system/global-pull-secret-syncer-pv8mz" Sep 08 10:07:29.151043 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:29.150959 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjrcr\" (UniqueName: \"kubernetes.io/projected/808f3bbd-c348-4c64-8ea7-e2e692594df8-kube-api-access-rjrcr\") pod \"network-check-target-rx2jz\" (UID: \"808f3bbd-c348-4c64-8ea7-e2e692594df8\") " pod="openshift-network-diagnostics/network-check-target-rx2jz" Sep 08 10:07:29.151106 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:29.151046 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 10:07:29.151106 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:29.151084 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 10:07:29.151106 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:29.151104 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 10:07:29.151220 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:29.151116 2568 projected.go:196] Error preparing data for projected volume kube-api-access-rjrcr for pod openshift-network-diagnostics/network-check-target-rx2jz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 10:07:29.151220 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:29.151120 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b608607f-8020-4d4e-9f43-b369b983a3b6-original-pull-secret podName:b608607f-8020-4d4e-9f43-b369b983a3b6 nodeName:}" failed. No retries permitted until 2026-09-08 10:08:01.151098694 +0000 UTC m=+65.195202747 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b608607f-8020-4d4e-9f43-b369b983a3b6-original-pull-secret") pod "global-pull-secret-syncer-pv8mz" (UID: "b608607f-8020-4d4e-9f43-b369b983a3b6") : object "kube-system"/"original-pull-secret" not registered Sep 08 10:07:29.151220 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:29.151161 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/808f3bbd-c348-4c64-8ea7-e2e692594df8-kube-api-access-rjrcr podName:808f3bbd-c348-4c64-8ea7-e2e692594df8 nodeName:}" failed. No retries permitted until 2026-09-08 10:08:01.151145836 +0000 UTC m=+65.195249886 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rjrcr" (UniqueName: "kubernetes.io/projected/808f3bbd-c348-4c64-8ea7-e2e692594df8-kube-api-access-rjrcr") pod "network-check-target-rx2jz" (UID: "808f3bbd-c348-4c64-8ea7-e2e692594df8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 10:07:29.398610 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:29.398582 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbs64" Sep 08 10:07:29.398765 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:29.398593 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx2jz" Sep 08 10:07:29.398765 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:29.398702 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv8mz" Sep 08 10:07:29.398865 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:29.398699 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mbs64" podUID="3b4e4397-1898-4201-9dcd-b5775ee1789c" Sep 08 10:07:29.398865 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:29.398770 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pv8mz" podUID="b608607f-8020-4d4e-9f43-b369b983a3b6" Sep 08 10:07:29.398865 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:29.398835 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rx2jz" podUID="808f3bbd-c348-4c64-8ea7-e2e692594df8" Sep 08 10:07:29.726403 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:29.726335 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-137.ec2.internal" event="NodeReady" Sep 08 10:07:29.726590 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:29.726483 2568 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 08 10:07:29.803309 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:29.803275 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dl46k"] Sep 08 10:07:29.816592 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:29.816571 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rhxl4"] Sep 08 10:07:29.816707 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:29.816688 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dl46k" Sep 08 10:07:29.820523 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:29.820503 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-t8hl6\"" Sep 08 10:07:29.820822 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:29.820801 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 08 10:07:29.820922 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:29.820867 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 08 10:07:29.837021 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:29.836998 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dl46k"] Sep 08 10:07:29.837250 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:29.837042 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rhxl4"] Sep 08 10:07:29.837250 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:29.837115 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rhxl4" Sep 08 10:07:29.840356 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:29.840336 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 08 10:07:29.841191 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:29.841035 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bpsj5\"" Sep 08 10:07:29.841191 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:29.840341 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 08 10:07:29.841191 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:29.840730 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 08 10:07:29.957932 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:29.957909 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpbwp\" (UniqueName: \"kubernetes.io/projected/0045c129-e95a-4d80-a1b6-c972ee592d76-kube-api-access-hpbwp\") pod \"dns-default-dl46k\" (UID: \"0045c129-e95a-4d80-a1b6-c972ee592d76\") " pod="openshift-dns/dns-default-dl46k" Sep 08 10:07:29.958074 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:29.958020 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0045c129-e95a-4d80-a1b6-c972ee592d76-metrics-tls\") pod \"dns-default-dl46k\" (UID: \"0045c129-e95a-4d80-a1b6-c972ee592d76\") " pod="openshift-dns/dns-default-dl46k" Sep 08 10:07:29.958141 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:29.958104 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bv8h9\" (UniqueName: \"kubernetes.io/projected/7bfb2025-199a-4235-a283-7d555387fd2c-kube-api-access-bv8h9\") pod \"ingress-canary-rhxl4\" (UID: \"7bfb2025-199a-4235-a283-7d555387fd2c\") " pod="openshift-ingress-canary/ingress-canary-rhxl4" Sep 08 10:07:29.958198 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:29.958137 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0045c129-e95a-4d80-a1b6-c972ee592d76-config-volume\") pod \"dns-default-dl46k\" (UID: \"0045c129-e95a-4d80-a1b6-c972ee592d76\") " pod="openshift-dns/dns-default-dl46k" Sep 08 10:07:29.958198 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:29.958169 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0045c129-e95a-4d80-a1b6-c972ee592d76-tmp-dir\") pod \"dns-default-dl46k\" (UID: \"0045c129-e95a-4d80-a1b6-c972ee592d76\") " pod="openshift-dns/dns-default-dl46k" Sep 08 10:07:29.958301 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:29.958282 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7bfb2025-199a-4235-a283-7d555387fd2c-cert\") pod \"ingress-canary-rhxl4\" (UID: \"7bfb2025-199a-4235-a283-7d555387fd2c\") " pod="openshift-ingress-canary/ingress-canary-rhxl4" Sep 08 10:07:30.059255 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:30.059198 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7bfb2025-199a-4235-a283-7d555387fd2c-cert\") pod \"ingress-canary-rhxl4\" (UID: \"7bfb2025-199a-4235-a283-7d555387fd2c\") " pod="openshift-ingress-canary/ingress-canary-rhxl4" Sep 08 10:07:30.059669 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:30.059247 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpbwp\" (UniqueName: \"kubernetes.io/projected/0045c129-e95a-4d80-a1b6-c972ee592d76-kube-api-access-hpbwp\") pod \"dns-default-dl46k\" (UID: \"0045c129-e95a-4d80-a1b6-c972ee592d76\") " pod="openshift-dns/dns-default-dl46k" Sep 08 10:07:30.059669 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:30.059455 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0045c129-e95a-4d80-a1b6-c972ee592d76-metrics-tls\") pod \"dns-default-dl46k\" (UID: \"0045c129-e95a-4d80-a1b6-c972ee592d76\") " pod="openshift-dns/dns-default-dl46k" Sep 08 10:07:30.059669 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:30.059502 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bv8h9\" (UniqueName: \"kubernetes.io/projected/7bfb2025-199a-4235-a283-7d555387fd2c-kube-api-access-bv8h9\") pod \"ingress-canary-rhxl4\" (UID: \"7bfb2025-199a-4235-a283-7d555387fd2c\") " pod="openshift-ingress-canary/ingress-canary-rhxl4" Sep 08 10:07:30.059669 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:30.059534 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0045c129-e95a-4d80-a1b6-c972ee592d76-config-volume\") pod \"dns-default-dl46k\" (UID: \"0045c129-e95a-4d80-a1b6-c972ee592d76\") " pod="openshift-dns/dns-default-dl46k" Sep 08 10:07:30.059669 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:30.059557 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0045c129-e95a-4d80-a1b6-c972ee592d76-tmp-dir\") pod \"dns-default-dl46k\" (UID: \"0045c129-e95a-4d80-a1b6-c972ee592d76\") " pod="openshift-dns/dns-default-dl46k" Sep 08 10:07:30.059669 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:30.059558 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 10:07:30.059669 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:30.059627 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0045c129-e95a-4d80-a1b6-c972ee592d76-metrics-tls podName:0045c129-e95a-4d80-a1b6-c972ee592d76 nodeName:}" failed. No retries permitted until 2026-09-08 10:07:30.559607333 +0000 UTC m=+34.603711394 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0045c129-e95a-4d80-a1b6-c972ee592d76-metrics-tls") pod "dns-default-dl46k" (UID: "0045c129-e95a-4d80-a1b6-c972ee592d76") : secret "dns-default-metrics-tls" not found Sep 08 10:07:30.060027 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:30.059818 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0045c129-e95a-4d80-a1b6-c972ee592d76-tmp-dir\") pod \"dns-default-dl46k\" (UID: \"0045c129-e95a-4d80-a1b6-c972ee592d76\") " pod="openshift-dns/dns-default-dl46k" Sep 08 10:07:30.060276 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:30.060239 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0045c129-e95a-4d80-a1b6-c972ee592d76-config-volume\") pod \"dns-default-dl46k\" (UID: \"0045c129-e95a-4d80-a1b6-c972ee592d76\") " pod="openshift-dns/dns-default-dl46k" Sep 08 10:07:30.064112 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:30.064091 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7bfb2025-199a-4235-a283-7d555387fd2c-cert\") pod \"ingress-canary-rhxl4\" (UID: \"7bfb2025-199a-4235-a283-7d555387fd2c\") " pod="openshift-ingress-canary/ingress-canary-rhxl4" Sep 08 10:07:30.069236 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:30.069214 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bv8h9\" (UniqueName: \"kubernetes.io/projected/7bfb2025-199a-4235-a283-7d555387fd2c-kube-api-access-bv8h9\") pod \"ingress-canary-rhxl4\" (UID: \"7bfb2025-199a-4235-a283-7d555387fd2c\") " pod="openshift-ingress-canary/ingress-canary-rhxl4" Sep 08 10:07:30.069574 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:30.069556 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpbwp\" (UniqueName: \"kubernetes.io/projected/0045c129-e95a-4d80-a1b6-c972ee592d76-kube-api-access-hpbwp\") pod \"dns-default-dl46k\" (UID: \"0045c129-e95a-4d80-a1b6-c972ee592d76\") " pod="openshift-dns/dns-default-dl46k" Sep 08 10:07:30.146310 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:30.146284 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rhxl4" Sep 08 10:07:30.485791 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:30.485613 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rhxl4"] Sep 08 10:07:30.490451 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:07:30.490404 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bfb2025_199a_4235_a283_7d555387fd2c.slice/crio-812806e9b6dd05454375b10869bac67d02d8d65ebf095aa06f480b67f8d826bb WatchSource:0}: Error finding container 812806e9b6dd05454375b10869bac67d02d8d65ebf095aa06f480b67f8d826bb: Status 404 returned error can't find the container with id 812806e9b6dd05454375b10869bac67d02d8d65ebf095aa06f480b67f8d826bb Sep 08 10:07:30.545216 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:30.545184 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rhxl4" event={"ID":"7bfb2025-199a-4235-a283-7d555387fd2c","Type":"ContainerStarted","Data":"812806e9b6dd05454375b10869bac67d02d8d65ebf095aa06f480b67f8d826bb"} Sep 08 10:07:30.564585 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:30.564562 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0045c129-e95a-4d80-a1b6-c972ee592d76-metrics-tls\") pod \"dns-default-dl46k\" (UID: \"0045c129-e95a-4d80-a1b6-c972ee592d76\") " pod="openshift-dns/dns-default-dl46k" Sep 08 10:07:30.564729 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:30.564713 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 10:07:30.564774 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:30.564763 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0045c129-e95a-4d80-a1b6-c972ee592d76-metrics-tls podName:0045c129-e95a-4d80-a1b6-c972ee592d76 nodeName:}" failed. No retries permitted until 2026-09-08 10:07:31.564748944 +0000 UTC m=+35.608852990 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0045c129-e95a-4d80-a1b6-c972ee592d76-metrics-tls") pod "dns-default-dl46k" (UID: "0045c129-e95a-4d80-a1b6-c972ee592d76") : secret "dns-default-metrics-tls" not found Sep 08 10:07:31.399396 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:31.399347 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv8mz" Sep 08 10:07:31.399874 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:31.399458 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx2jz" Sep 08 10:07:31.399874 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:31.399568 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbs64" Sep 08 10:07:31.402648 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:31.402626 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 08 10:07:31.403018 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:31.402808 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 08 10:07:31.403018 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:31.402854 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 08 10:07:31.403424 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:31.403391 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tm4jr\"" Sep 08 10:07:31.403800 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:31.403635 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 08 10:07:31.403890 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:31.403808 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-755cm\"" Sep 08 10:07:31.571781 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:31.571745 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0045c129-e95a-4d80-a1b6-c972ee592d76-metrics-tls\") pod \"dns-default-dl46k\" (UID: \"0045c129-e95a-4d80-a1b6-c972ee592d76\") " pod="openshift-dns/dns-default-dl46k" Sep 08 10:07:31.571957 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:31.571898 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 10:07:31.572010 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:31.571970 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0045c129-e95a-4d80-a1b6-c972ee592d76-metrics-tls podName:0045c129-e95a-4d80-a1b6-c972ee592d76 nodeName:}" failed. No retries permitted until 2026-09-08 10:07:33.571942295 +0000 UTC m=+37.616046428 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0045c129-e95a-4d80-a1b6-c972ee592d76-metrics-tls") pod "dns-default-dl46k" (UID: "0045c129-e95a-4d80-a1b6-c972ee592d76") : secret "dns-default-metrics-tls" not found Sep 08 10:07:33.586437 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:33.586389 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0045c129-e95a-4d80-a1b6-c972ee592d76-metrics-tls\") pod \"dns-default-dl46k\" (UID: \"0045c129-e95a-4d80-a1b6-c972ee592d76\") " pod="openshift-dns/dns-default-dl46k" Sep 08 10:07:33.586915 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:33.586561 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 10:07:33.586915 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:33.586638 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0045c129-e95a-4d80-a1b6-c972ee592d76-metrics-tls podName:0045c129-e95a-4d80-a1b6-c972ee592d76 nodeName:}" failed. No retries permitted until 2026-09-08 10:07:37.586615389 +0000 UTC m=+41.630719450 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0045c129-e95a-4d80-a1b6-c972ee592d76-metrics-tls") pod "dns-default-dl46k" (UID: "0045c129-e95a-4d80-a1b6-c972ee592d76") : secret "dns-default-metrics-tls" not found Sep 08 10:07:34.556492 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:34.556405 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rhxl4" event={"ID":"7bfb2025-199a-4235-a283-7d555387fd2c","Type":"ContainerStarted","Data":"18a9bdf5ca356d26438d2529341de3efebc765b95d27d788109ad931e7e84374"} Sep 08 10:07:34.559120 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:34.559096 2568 generic.go:356] "Generic (PLEG): container finished" podID="9a781adb-4df3-440c-965a-1aa5e72f7271" containerID="0d5692530eb6cf53eb333fe8bba8edf9c5bc6d22af02958a183a55175fd02740" exitCode=0 Sep 08 10:07:34.559120 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:34.559127 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w5t97" event={"ID":"9a781adb-4df3-440c-965a-1aa5e72f7271","Type":"ContainerDied","Data":"0d5692530eb6cf53eb333fe8bba8edf9c5bc6d22af02958a183a55175fd02740"} Sep 08 10:07:34.576974 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:34.576931 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rhxl4" podStartSLOduration=1.982975103 podStartE2EDuration="5.57691788s" totalImagesPullingTime="3.593942777s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:07:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:07:34.576221124 +0000 UTC m=+38.620325203" watchObservedRunningTime="2026-09-08 10:07:34.57691788 +0000 UTC m=+38.621021945" Sep 08 10:07:35.567688 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:35.567651 2568 generic.go:356] "Generic (PLEG): container finished" podID="9a781adb-4df3-440c-965a-1aa5e72f7271" containerID="02569261f81230db68a40417d950fb6dc747ae8960994602fa40eb1aa4b3d369" exitCode=0 Sep 08 10:07:35.568084 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:35.567711 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w5t97" event={"ID":"9a781adb-4df3-440c-965a-1aa5e72f7271","Type":"ContainerDied","Data":"02569261f81230db68a40417d950fb6dc747ae8960994602fa40eb1aa4b3d369"} Sep 08 10:07:36.572834 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:36.572801 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w5t97" event={"ID":"9a781adb-4df3-440c-965a-1aa5e72f7271","Type":"ContainerStarted","Data":"4499870d8531f591e740fa0581e3a1d8d2f663267f9fc1c4e67bd0b3dda6664c"} Sep 08 10:07:36.596277 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:36.596234 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-w5t97" podStartSLOduration=6.415424061 podStartE2EDuration="40.59622017s" totalImagesPullingTime="34.180796109s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 10:06:56 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:07:36.594645634 +0000 UTC m=+40.638749721" watchObservedRunningTime="2026-09-08 10:07:36.59622017 +0000 UTC m=+40.640324238" Sep 08 10:07:37.613265 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:37.613236 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0045c129-e95a-4d80-a1b6-c972ee592d76-metrics-tls\") pod \"dns-default-dl46k\" (UID: \"0045c129-e95a-4d80-a1b6-c972ee592d76\") " pod="openshift-dns/dns-default-dl46k" Sep 08 10:07:37.613610 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:37.613330 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 10:07:37.613610 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:37.613378 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0045c129-e95a-4d80-a1b6-c972ee592d76-metrics-tls podName:0045c129-e95a-4d80-a1b6-c972ee592d76 nodeName:}" failed. No retries permitted until 2026-09-08 10:07:45.613365049 +0000 UTC m=+49.657469098 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0045c129-e95a-4d80-a1b6-c972ee592d76-metrics-tls") pod "dns-default-dl46k" (UID: "0045c129-e95a-4d80-a1b6-c972ee592d76") : secret "dns-default-metrics-tls" not found Sep 08 10:07:45.662999 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:45.662965 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0045c129-e95a-4d80-a1b6-c972ee592d76-metrics-tls\") pod \"dns-default-dl46k\" (UID: \"0045c129-e95a-4d80-a1b6-c972ee592d76\") " pod="openshift-dns/dns-default-dl46k" Sep 08 10:07:45.663369 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:45.663114 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 10:07:45.663369 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:07:45.663172 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0045c129-e95a-4d80-a1b6-c972ee592d76-metrics-tls podName:0045c129-e95a-4d80-a1b6-c972ee592d76 nodeName:}" failed. No retries permitted until 2026-09-08 10:08:01.663156796 +0000 UTC m=+65.707260847 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0045c129-e95a-4d80-a1b6-c972ee592d76-metrics-tls") pod "dns-default-dl46k" (UID: "0045c129-e95a-4d80-a1b6-c972ee592d76") : secret "dns-default-metrics-tls" not found Sep 08 10:07:54.545862 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:07:54.545834 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-k296b" Sep 08 10:08:01.069868 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:08:01.069828 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b4e4397-1898-4201-9dcd-b5775ee1789c-metrics-certs\") pod \"network-metrics-daemon-mbs64\" (UID: \"3b4e4397-1898-4201-9dcd-b5775ee1789c\") " pod="openshift-multus/network-metrics-daemon-mbs64" Sep 08 10:08:01.072280 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:08:01.072263 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 08 10:08:01.080180 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:08:01.080164 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 08 10:08:01.080241 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:08:01.080215 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b4e4397-1898-4201-9dcd-b5775ee1789c-metrics-certs podName:3b4e4397-1898-4201-9dcd-b5775ee1789c nodeName:}" failed. No retries permitted until 2026-09-08 10:09:05.080199724 +0000 UTC m=+129.124303771 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b4e4397-1898-4201-9dcd-b5775ee1789c-metrics-certs") pod "network-metrics-daemon-mbs64" (UID: "3b4e4397-1898-4201-9dcd-b5775ee1789c") : secret "metrics-daemon-secret" not found Sep 08 10:08:01.170175 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:08:01.170150 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b608607f-8020-4d4e-9f43-b369b983a3b6-original-pull-secret\") pod \"global-pull-secret-syncer-pv8mz\" (UID: \"b608607f-8020-4d4e-9f43-b369b983a3b6\") " pod="kube-system/global-pull-secret-syncer-pv8mz" Sep 08 10:08:01.170255 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:08:01.170204 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjrcr\" (UniqueName: \"kubernetes.io/projected/808f3bbd-c348-4c64-8ea7-e2e692594df8-kube-api-access-rjrcr\") pod \"network-check-target-rx2jz\" (UID: \"808f3bbd-c348-4c64-8ea7-e2e692594df8\") " pod="openshift-network-diagnostics/network-check-target-rx2jz" Sep 08 10:08:01.172950 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:08:01.172935 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 08 10:08:01.173012 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:08:01.172999 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 08 10:08:01.182773 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:08:01.182751 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 08 10:08:01.183950 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:08:01.183923 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b608607f-8020-4d4e-9f43-b369b983a3b6-original-pull-secret\") pod \"global-pull-secret-syncer-pv8mz\" (UID: \"b608607f-8020-4d4e-9f43-b369b983a3b6\") " pod="kube-system/global-pull-secret-syncer-pv8mz" Sep 08 10:08:01.193828 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:08:01.193811 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjrcr\" (UniqueName: \"kubernetes.io/projected/808f3bbd-c348-4c64-8ea7-e2e692594df8-kube-api-access-rjrcr\") pod \"network-check-target-rx2jz\" (UID: \"808f3bbd-c348-4c64-8ea7-e2e692594df8\") " pod="openshift-network-diagnostics/network-check-target-rx2jz" Sep 08 10:08:01.412866 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:08:01.412846 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv8mz" Sep 08 10:08:01.432984 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:08:01.432965 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-755cm\"" Sep 08 10:08:01.440990 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:08:01.440975 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx2jz" Sep 08 10:08:01.674836 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:08:01.674781 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0045c129-e95a-4d80-a1b6-c972ee592d76-metrics-tls\") pod \"dns-default-dl46k\" (UID: \"0045c129-e95a-4d80-a1b6-c972ee592d76\") " pod="openshift-dns/dns-default-dl46k" Sep 08 10:08:01.674951 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:08:01.674895 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 10:08:01.674951 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:08:01.674943 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0045c129-e95a-4d80-a1b6-c972ee592d76-metrics-tls podName:0045c129-e95a-4d80-a1b6-c972ee592d76 nodeName:}" failed. No retries permitted until 2026-09-08 10:08:33.674929435 +0000 UTC m=+97.719033481 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0045c129-e95a-4d80-a1b6-c972ee592d76-metrics-tls") pod "dns-default-dl46k" (UID: "0045c129-e95a-4d80-a1b6-c972ee592d76") : secret "dns-default-metrics-tls" not found Sep 08 10:08:01.679684 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:08:01.679608 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pv8mz"] Sep 08 10:08:01.682043 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:08:01.682020 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb608607f_8020_4d4e_9f43_b369b983a3b6.slice/crio-6aec6963c019069c8b6fe651b44cb4f6e5531d0f03470177da48640e07401c21 WatchSource:0}: Error finding container 6aec6963c019069c8b6fe651b44cb4f6e5531d0f03470177da48640e07401c21: Status 404 returned error can't find the container with id 6aec6963c019069c8b6fe651b44cb4f6e5531d0f03470177da48640e07401c21 Sep 08 10:08:01.709382 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:08:01.709361 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rx2jz"] Sep 08 10:08:01.711967 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:08:01.711945 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod808f3bbd_c348_4c64_8ea7_e2e692594df8.slice/crio-a13cf0c40f0dc7aad6514e430a185583a407582cb892f39f513dfe2d4a3b5b3c WatchSource:0}: Error finding container a13cf0c40f0dc7aad6514e430a185583a407582cb892f39f513dfe2d4a3b5b3c: Status 404 returned error can't find the container with id a13cf0c40f0dc7aad6514e430a185583a407582cb892f39f513dfe2d4a3b5b3c Sep 08 10:08:02.632399 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:08:02.632362 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rx2jz" event={"ID":"808f3bbd-c348-4c64-8ea7-e2e692594df8","Type":"ContainerStarted","Data":"a13cf0c40f0dc7aad6514e430a185583a407582cb892f39f513dfe2d4a3b5b3c"} Sep 08 10:08:02.633763 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:08:02.633729 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pv8mz" event={"ID":"b608607f-8020-4d4e-9f43-b369b983a3b6","Type":"ContainerStarted","Data":"6aec6963c019069c8b6fe651b44cb4f6e5531d0f03470177da48640e07401c21"} Sep 08 10:08:04.642738 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:08:04.642494 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rx2jz" event={"ID":"808f3bbd-c348-4c64-8ea7-e2e692594df8","Type":"ContainerStarted","Data":"4538ca82d732c0e573dc203c7fc59dddd9d900122b400566d654c2394e81360d"} Sep 08 10:08:04.643244 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:08:04.643220 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rx2jz" Sep 08 10:08:04.658831 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:08:04.658744 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-rx2jz" podStartSLOduration=65.908334381 podStartE2EDuration="1m8.65872606s" totalImagesPullingTime="2.750391679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:06:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:08:04.658436054 +0000 UTC m=+68.702540130" watchObservedRunningTime="2026-09-08 10:08:04.65872606 +0000 UTC m=+68.702830128" Sep 08 10:08:08.653150 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:08:08.653117 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pv8mz" event={"ID":"b608607f-8020-4d4e-9f43-b369b983a3b6","Type":"ContainerStarted","Data":"9869f1d86f9a5af52cdf76a847647ac3ae50123bdd6dcdbc9ca67992c3b14371"} Sep 08 10:08:08.667024 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:08:08.666980 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-pv8mz" podStartSLOduration=66.221114078 podStartE2EDuration="1m12.666966593s" totalImagesPullingTime="6.445852515s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 10:06:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:08:08.666554869 +0000 UTC m=+72.710658957" watchObservedRunningTime="2026-09-08 10:08:08.666966593 +0000 UTC m=+72.711070688" Sep 08 10:08:33.680823 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:08:33.680738 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0045c129-e95a-4d80-a1b6-c972ee592d76-metrics-tls\") pod \"dns-default-dl46k\" (UID: \"0045c129-e95a-4d80-a1b6-c972ee592d76\") " pod="openshift-dns/dns-default-dl46k" Sep 08 10:08:33.681155 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:08:33.680885 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 10:08:33.681155 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:08:33.680958 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0045c129-e95a-4d80-a1b6-c972ee592d76-metrics-tls podName:0045c129-e95a-4d80-a1b6-c972ee592d76 nodeName:}" failed. No retries permitted until 2026-09-08 10:09:37.68094182 +0000 UTC m=+161.725045866 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0045c129-e95a-4d80-a1b6-c972ee592d76-metrics-tls") pod "dns-default-dl46k" (UID: "0045c129-e95a-4d80-a1b6-c972ee592d76") : secret "dns-default-metrics-tls" not found Sep 08 10:08:36.649218 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:08:36.649185 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-rx2jz" Sep 08 10:09:01.503863 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.503828 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-524nf"] Sep 08 10:09:01.505682 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.505663 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-524nf" Sep 08 10:09:01.509250 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.509228 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-v4xdd\"" Sep 08 10:09:01.509429 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.509392 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 08 10:09:01.509499 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.509442 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Sep 08 10:09:01.510361 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.510342 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 08 10:09:01.510441 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.510381 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Sep 08 10:09:01.519521 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.519500 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-524nf"] Sep 08 10:09:01.519857 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.519842 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Sep 08 10:09:01.553865 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.553841 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccdd6\" (UniqueName: \"kubernetes.io/projected/852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a-kube-api-access-ccdd6\") pod \"insights-operator-76bcd6ccf5-524nf\" (UID: \"852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a\") " pod="openshift-insights/insights-operator-76bcd6ccf5-524nf" Sep 08 10:09:01.554028 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.553890 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-524nf\" (UID: \"852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a\") " pod="openshift-insights/insights-operator-76bcd6ccf5-524nf" Sep 08 10:09:01.554028 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.553910 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a-tmp\") pod \"insights-operator-76bcd6ccf5-524nf\" (UID: \"852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a\") " pod="openshift-insights/insights-operator-76bcd6ccf5-524nf" Sep 08 10:09:01.554028 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.554004 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a-snapshots\") pod \"insights-operator-76bcd6ccf5-524nf\" (UID: \"852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a\") " pod="openshift-insights/insights-operator-76bcd6ccf5-524nf" Sep 08 10:09:01.554156 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.554039 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-524nf\" (UID: \"852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a\") " pod="openshift-insights/insights-operator-76bcd6ccf5-524nf" Sep 08 10:09:01.554156 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.554065 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a-serving-cert\") pod \"insights-operator-76bcd6ccf5-524nf\" (UID: \"852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a\") " pod="openshift-insights/insights-operator-76bcd6ccf5-524nf" Sep 08 10:09:01.595620 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.595597 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-bjv7p"] Sep 08 10:09:01.597290 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.597277 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bjv7p" Sep 08 10:09:01.600115 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.600095 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-mzztq\"" Sep 08 10:09:01.600270 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.600252 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Sep 08 10:09:01.600362 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.600347 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Sep 08 10:09:01.600482 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.600465 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Sep 08 10:09:01.600546 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.600474 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Sep 08 10:09:01.603116 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.602874 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-65cd7bbbb6-k5m7p"] Sep 08 10:09:01.604581 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.604542 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qs5nv"] Sep 08 10:09:01.604670 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.604658 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-65cd7bbbb6-k5m7p" Sep 08 10:09:01.606227 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.606211 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6d54b6c6c4-bxfp6"] Sep 08 10:09:01.606320 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.606308 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qs5nv" Sep 08 10:09:01.607721 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.607706 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" Sep 08 10:09:01.642438 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.641086 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 08 10:09:01.642438 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.641286 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Sep 08 10:09:01.642438 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.641574 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 08 10:09:01.642438 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.641755 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Sep 08 10:09:01.642438 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.641905 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Sep 08 10:09:01.642438 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.642061 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Sep 08 10:09:01.642438 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.642210 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Sep 08 10:09:01.642438 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.642366 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Sep 08 10:09:01.642873 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.642558 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Sep 08 10:09:01.642873 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.642758 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 08 10:09:01.642978 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.642936 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 08 10:09:01.645214 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.643117 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-wd8xl\"" Sep 08 10:09:01.645214 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.643295 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-6htx2\"" Sep 08 10:09:01.645396 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.645334 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-ssdtn\"" Sep 08 10:09:01.647834 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.647813 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 08 10:09:01.648395 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.647989 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-bjv7p"] Sep 08 10:09:01.648610 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.648596 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qs5nv"] Sep 08 10:09:01.648738 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.648597 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 08 10:09:01.649384 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.649363 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-65cd7bbbb6-k5m7p"] Sep 08 10:09:01.654523 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.654505 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d28e6ab-62c9-45a3-ae14-18623958e9ef-service-ca-bundle\") pod \"router-default-65cd7bbbb6-k5m7p\" (UID: \"7d28e6ab-62c9-45a3-ae14-18623958e9ef\") " pod="openshift-ingress/router-default-65cd7bbbb6-k5m7p" Sep 08 10:09:01.654598 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.654531 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1373d83d-e33c-4993-bfe5-7ff0b9241a79-config\") pod \"service-ca-operator-859f94c4b-bjv7p\" (UID: \"1373d83d-e33c-4993-bfe5-7ff0b9241a79\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bjv7p" Sep 08 10:09:01.654598 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.654547 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-trusted-ca\") pod \"image-registry-6d54b6c6c4-bxfp6\" (UID: \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\") " pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" Sep 08 10:09:01.654598 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.654568 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-524nf\" (UID: \"852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a\") " pod="openshift-insights/insights-operator-76bcd6ccf5-524nf" Sep 08 10:09:01.654598 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.654583 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a-serving-cert\") pod \"insights-operator-76bcd6ccf5-524nf\" (UID: \"852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a\") " pod="openshift-insights/insights-operator-76bcd6ccf5-524nf" Sep 08 10:09:01.654763 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.654614 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccdd6\" (UniqueName: \"kubernetes.io/projected/852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a-kube-api-access-ccdd6\") pod \"insights-operator-76bcd6ccf5-524nf\" (UID: \"852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a\") " pod="openshift-insights/insights-operator-76bcd6ccf5-524nf" Sep 08 10:09:01.654763 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.654645 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-registry-tls\") pod \"image-registry-6d54b6c6c4-bxfp6\" (UID: \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\") " pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" Sep 08 10:09:01.654763 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.654671 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d28e6ab-62c9-45a3-ae14-18623958e9ef-metrics-certs\") pod \"router-default-65cd7bbbb6-k5m7p\" (UID: \"7d28e6ab-62c9-45a3-ae14-18623958e9ef\") " pod="openshift-ingress/router-default-65cd7bbbb6-k5m7p" Sep 08 10:09:01.654763 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.654738 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84lkj\" (UniqueName: \"kubernetes.io/projected/df6fbf45-0c71-4456-a95c-a5d5b24aa109-kube-api-access-84lkj\") pod \"cluster-samples-operator-7cdcfcf76c-qs5nv\" (UID: \"df6fbf45-0c71-4456-a95c-a5d5b24aa109\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qs5nv" Sep 08 10:09:01.654903 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.654768 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-524nf\" (UID: \"852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a\") " pod="openshift-insights/insights-operator-76bcd6ccf5-524nf" Sep 08 10:09:01.654903 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.654785 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-registry-certificates\") pod \"image-registry-6d54b6c6c4-bxfp6\" (UID: \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\") " pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" Sep 08 10:09:01.654903 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.654806 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df6fbf45-0c71-4456-a95c-a5d5b24aa109-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-qs5nv\" (UID: \"df6fbf45-0c71-4456-a95c-a5d5b24aa109\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qs5nv" Sep 08 10:09:01.654903 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.654832 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-ca-trust-extracted\") pod \"image-registry-6d54b6c6c4-bxfp6\" (UID: \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\") " pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" Sep 08 10:09:01.654903 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.654862 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7d28e6ab-62c9-45a3-ae14-18623958e9ef-default-certificate\") pod \"router-default-65cd7bbbb6-k5m7p\" (UID: \"7d28e6ab-62c9-45a3-ae14-18623958e9ef\") " pod="openshift-ingress/router-default-65cd7bbbb6-k5m7p" Sep 08 10:09:01.655099 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.654907 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvnmt\" (UniqueName: \"kubernetes.io/projected/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-kube-api-access-zvnmt\") pod \"image-registry-6d54b6c6c4-bxfp6\" (UID: \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\") " pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" Sep 08 10:09:01.655099 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.654934 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-installation-pull-secrets\") pod \"image-registry-6d54b6c6c4-bxfp6\" (UID: \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\") " pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" Sep 08 10:09:01.655099 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.654958 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7d28e6ab-62c9-45a3-ae14-18623958e9ef-stats-auth\") pod \"router-default-65cd7bbbb6-k5m7p\" (UID: \"7d28e6ab-62c9-45a3-ae14-18623958e9ef\") " pod="openshift-ingress/router-default-65cd7bbbb6-k5m7p" Sep 08 10:09:01.655099 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.655007 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1373d83d-e33c-4993-bfe5-7ff0b9241a79-serving-cert\") pod \"service-ca-operator-859f94c4b-bjv7p\" (UID: \"1373d83d-e33c-4993-bfe5-7ff0b9241a79\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bjv7p" Sep 08 10:09:01.655099 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.655031 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5sbw\" (UniqueName: \"kubernetes.io/projected/1373d83d-e33c-4993-bfe5-7ff0b9241a79-kube-api-access-m5sbw\") pod \"service-ca-operator-859f94c4b-bjv7p\" (UID: \"1373d83d-e33c-4993-bfe5-7ff0b9241a79\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bjv7p" Sep 08 10:09:01.655099 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.655078 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-image-registry-private-configuration\") pod \"image-registry-6d54b6c6c4-bxfp6\" (UID: \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\") " pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" Sep 08 10:09:01.655360 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.655104 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-bound-sa-token\") pod \"image-registry-6d54b6c6c4-bxfp6\" (UID: \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\") " pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" Sep 08 10:09:01.655360 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.655138 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a-tmp\") pod \"insights-operator-76bcd6ccf5-524nf\" (UID: \"852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a\") " pod="openshift-insights/insights-operator-76bcd6ccf5-524nf" Sep 08 10:09:01.655360 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.655202 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x94wj\" (UniqueName: \"kubernetes.io/projected/7d28e6ab-62c9-45a3-ae14-18623958e9ef-kube-api-access-x94wj\") pod \"router-default-65cd7bbbb6-k5m7p\" (UID: \"7d28e6ab-62c9-45a3-ae14-18623958e9ef\") " pod="openshift-ingress/router-default-65cd7bbbb6-k5m7p" Sep 08 10:09:01.655360 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.655235 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a-snapshots\") pod \"insights-operator-76bcd6ccf5-524nf\" (UID: \"852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a\") " pod="openshift-insights/insights-operator-76bcd6ccf5-524nf" Sep 08 10:09:01.655547 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.655390 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-524nf\" (UID: \"852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a\") " pod="openshift-insights/insights-operator-76bcd6ccf5-524nf" Sep 08 10:09:01.655547 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.655436 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a-tmp\") pod \"insights-operator-76bcd6ccf5-524nf\" (UID: \"852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a\") " pod="openshift-insights/insights-operator-76bcd6ccf5-524nf" Sep 08 10:09:01.655547 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.655455 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-524nf\" (UID: \"852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a\") " pod="openshift-insights/insights-operator-76bcd6ccf5-524nf" Sep 08 10:09:01.655808 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.655787 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a-snapshots\") pod \"insights-operator-76bcd6ccf5-524nf\" (UID: \"852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a\") " pod="openshift-insights/insights-operator-76bcd6ccf5-524nf" Sep 08 10:09:01.656917 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.656900 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a-serving-cert\") pod \"insights-operator-76bcd6ccf5-524nf\" (UID: \"852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a\") " pod="openshift-insights/insights-operator-76bcd6ccf5-524nf" Sep 08 10:09:01.665002 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.664982 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d54b6c6c4-bxfp6"] Sep 08 10:09:01.679782 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.679766 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccdd6\" (UniqueName: \"kubernetes.io/projected/852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a-kube-api-access-ccdd6\") pod \"insights-operator-76bcd6ccf5-524nf\" (UID: \"852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a\") " pod="openshift-insights/insights-operator-76bcd6ccf5-524nf" Sep 08 10:09:01.755798 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.755732 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7d28e6ab-62c9-45a3-ae14-18623958e9ef-default-certificate\") pod \"router-default-65cd7bbbb6-k5m7p\" (UID: \"7d28e6ab-62c9-45a3-ae14-18623958e9ef\") " pod="openshift-ingress/router-default-65cd7bbbb6-k5m7p" Sep 08 10:09:01.755798 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.755776 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvnmt\" (UniqueName: \"kubernetes.io/projected/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-kube-api-access-zvnmt\") pod \"image-registry-6d54b6c6c4-bxfp6\" (UID: \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\") " pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" Sep 08 10:09:01.755946 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.755804 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-installation-pull-secrets\") pod \"image-registry-6d54b6c6c4-bxfp6\" (UID: \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\") " pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" Sep 08 10:09:01.755946 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.755826 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7d28e6ab-62c9-45a3-ae14-18623958e9ef-stats-auth\") pod \"router-default-65cd7bbbb6-k5m7p\" (UID: \"7d28e6ab-62c9-45a3-ae14-18623958e9ef\") " pod="openshift-ingress/router-default-65cd7bbbb6-k5m7p" Sep 08 10:09:01.755946 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.755853 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1373d83d-e33c-4993-bfe5-7ff0b9241a79-serving-cert\") pod \"service-ca-operator-859f94c4b-bjv7p\" (UID: \"1373d83d-e33c-4993-bfe5-7ff0b9241a79\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bjv7p" Sep 08 10:09:01.755946 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.755876 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5sbw\" (UniqueName: \"kubernetes.io/projected/1373d83d-e33c-4993-bfe5-7ff0b9241a79-kube-api-access-m5sbw\") pod \"service-ca-operator-859f94c4b-bjv7p\" (UID: \"1373d83d-e33c-4993-bfe5-7ff0b9241a79\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bjv7p" Sep 08 10:09:01.755946 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.755917 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-image-registry-private-configuration\") pod \"image-registry-6d54b6c6c4-bxfp6\" (UID: \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\") " pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" Sep 08 10:09:01.755946 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.755942 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-bound-sa-token\") pod \"image-registry-6d54b6c6c4-bxfp6\" (UID: \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\") " pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" Sep 08 10:09:01.756250 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.755989 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x94wj\" (UniqueName: \"kubernetes.io/projected/7d28e6ab-62c9-45a3-ae14-18623958e9ef-kube-api-access-x94wj\") pod \"router-default-65cd7bbbb6-k5m7p\" (UID: \"7d28e6ab-62c9-45a3-ae14-18623958e9ef\") " pod="openshift-ingress/router-default-65cd7bbbb6-k5m7p" Sep 08 10:09:01.756250 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.756018 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d28e6ab-62c9-45a3-ae14-18623958e9ef-service-ca-bundle\") pod \"router-default-65cd7bbbb6-k5m7p\" (UID: \"7d28e6ab-62c9-45a3-ae14-18623958e9ef\") " pod="openshift-ingress/router-default-65cd7bbbb6-k5m7p" Sep 08 10:09:01.756250 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.756056 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1373d83d-e33c-4993-bfe5-7ff0b9241a79-config\") pod \"service-ca-operator-859f94c4b-bjv7p\" (UID: \"1373d83d-e33c-4993-bfe5-7ff0b9241a79\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bjv7p" Sep 08 10:09:01.756250 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.756079 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-trusted-ca\") pod \"image-registry-6d54b6c6c4-bxfp6\" (UID: \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\") " pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" Sep 08 10:09:01.756250 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.756109 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-registry-tls\") pod \"image-registry-6d54b6c6c4-bxfp6\" (UID: \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\") " pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" Sep 08 10:09:01.756250 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.756131 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d28e6ab-62c9-45a3-ae14-18623958e9ef-metrics-certs\") pod \"router-default-65cd7bbbb6-k5m7p\" (UID: \"7d28e6ab-62c9-45a3-ae14-18623958e9ef\") " pod="openshift-ingress/router-default-65cd7bbbb6-k5m7p" Sep 08 10:09:01.756250 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:01.756223 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 08 10:09:01.756611 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:01.756275 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d28e6ab-62c9-45a3-ae14-18623958e9ef-metrics-certs podName:7d28e6ab-62c9-45a3-ae14-18623958e9ef nodeName:}" failed. No retries permitted until 2026-09-08 10:09:02.25625819 +0000 UTC m=+126.300362255 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d28e6ab-62c9-45a3-ae14-18623958e9ef-metrics-certs") pod "router-default-65cd7bbbb6-k5m7p" (UID: "7d28e6ab-62c9-45a3-ae14-18623958e9ef") : secret "router-metrics-certs-default" not found Sep 08 10:09:01.757278 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:01.756989 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7d28e6ab-62c9-45a3-ae14-18623958e9ef-service-ca-bundle podName:7d28e6ab-62c9-45a3-ae14-18623958e9ef nodeName:}" failed. No retries permitted until 2026-09-08 10:09:02.25697071 +0000 UTC m=+126.301074781 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7d28e6ab-62c9-45a3-ae14-18623958e9ef-service-ca-bundle") pod "router-default-65cd7bbbb6-k5m7p" (UID: "7d28e6ab-62c9-45a3-ae14-18623958e9ef") : configmap references non-existent config key: service-ca.crt Sep 08 10:09:01.757278 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.757009 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1373d83d-e33c-4993-bfe5-7ff0b9241a79-config\") pod \"service-ca-operator-859f94c4b-bjv7p\" (UID: \"1373d83d-e33c-4993-bfe5-7ff0b9241a79\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bjv7p" Sep 08 10:09:01.757278 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:01.757069 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 08 10:09:01.757278 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.757070 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84lkj\" (UniqueName: \"kubernetes.io/projected/df6fbf45-0c71-4456-a95c-a5d5b24aa109-kube-api-access-84lkj\") pod \"cluster-samples-operator-7cdcfcf76c-qs5nv\" (UID: \"df6fbf45-0c71-4456-a95c-a5d5b24aa109\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qs5nv" Sep 08 10:09:01.757278 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:01.757082 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d54b6c6c4-bxfp6: secret "image-registry-tls" not found Sep 08 10:09:01.757278 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.757110 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-registry-certificates\") pod \"image-registry-6d54b6c6c4-bxfp6\" (UID: \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\") " pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" Sep 08 10:09:01.757278 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:01.757122 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-registry-tls podName:4c12bdd1-d3f2-4659-be40-e653c94d7cbe nodeName:}" failed. No retries permitted until 2026-09-08 10:09:02.257106826 +0000 UTC m=+126.301210879 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-registry-tls") pod "image-registry-6d54b6c6c4-bxfp6" (UID: "4c12bdd1-d3f2-4659-be40-e653c94d7cbe") : secret "image-registry-tls" not found Sep 08 10:09:01.757278 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.757161 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df6fbf45-0c71-4456-a95c-a5d5b24aa109-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-qs5nv\" (UID: \"df6fbf45-0c71-4456-a95c-a5d5b24aa109\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qs5nv" Sep 08 10:09:01.757278 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.757190 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-ca-trust-extracted\") pod \"image-registry-6d54b6c6c4-bxfp6\" (UID: \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\") " pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" Sep 08 10:09:01.757773 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.757454 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-trusted-ca\") pod \"image-registry-6d54b6c6c4-bxfp6\" (UID: \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\") " pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" Sep 08 10:09:01.757773 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.757555 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-registry-certificates\") pod \"image-registry-6d54b6c6c4-bxfp6\" (UID: \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\") " pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" Sep 08 10:09:01.757773 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:01.757667 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 08 10:09:01.757773 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:01.757759 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df6fbf45-0c71-4456-a95c-a5d5b24aa109-samples-operator-tls podName:df6fbf45-0c71-4456-a95c-a5d5b24aa109 nodeName:}" failed. No retries permitted until 2026-09-08 10:09:02.257740978 +0000 UTC m=+126.301845038 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/df6fbf45-0c71-4456-a95c-a5d5b24aa109-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-qs5nv" (UID: "df6fbf45-0c71-4456-a95c-a5d5b24aa109") : secret "samples-operator-tls" not found Sep 08 10:09:01.757979 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.757934 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-ca-trust-extracted\") pod \"image-registry-6d54b6c6c4-bxfp6\" (UID: \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\") " pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" Sep 08 10:09:01.758557 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.758530 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-installation-pull-secrets\") pod \"image-registry-6d54b6c6c4-bxfp6\" (UID: \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\") " pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" Sep 08 10:09:01.758802 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.758777 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1373d83d-e33c-4993-bfe5-7ff0b9241a79-serving-cert\") pod \"service-ca-operator-859f94c4b-bjv7p\" (UID: \"1373d83d-e33c-4993-bfe5-7ff0b9241a79\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bjv7p" Sep 08 10:09:01.758867 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.758808 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7d28e6ab-62c9-45a3-ae14-18623958e9ef-default-certificate\") pod \"router-default-65cd7bbbb6-k5m7p\" (UID: \"7d28e6ab-62c9-45a3-ae14-18623958e9ef\") " pod="openshift-ingress/router-default-65cd7bbbb6-k5m7p" Sep 08 10:09:01.758906 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.758879 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-image-registry-private-configuration\") pod \"image-registry-6d54b6c6c4-bxfp6\" (UID: \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\") " pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" Sep 08 10:09:01.758942 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.758911 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7d28e6ab-62c9-45a3-ae14-18623958e9ef-stats-auth\") pod \"router-default-65cd7bbbb6-k5m7p\" (UID: \"7d28e6ab-62c9-45a3-ae14-18623958e9ef\") " pod="openshift-ingress/router-default-65cd7bbbb6-k5m7p" Sep 08 10:09:01.787619 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.787592 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5sbw\" (UniqueName: \"kubernetes.io/projected/1373d83d-e33c-4993-bfe5-7ff0b9241a79-kube-api-access-m5sbw\") pod \"service-ca-operator-859f94c4b-bjv7p\" (UID: \"1373d83d-e33c-4993-bfe5-7ff0b9241a79\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bjv7p" Sep 08 10:09:01.796172 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.796151 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvnmt\" (UniqueName: \"kubernetes.io/projected/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-kube-api-access-zvnmt\") pod \"image-registry-6d54b6c6c4-bxfp6\" (UID: \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\") " pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" Sep 08 10:09:01.798361 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.798339 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x94wj\" (UniqueName: \"kubernetes.io/projected/7d28e6ab-62c9-45a3-ae14-18623958e9ef-kube-api-access-x94wj\") pod \"router-default-65cd7bbbb6-k5m7p\" (UID: \"7d28e6ab-62c9-45a3-ae14-18623958e9ef\") " pod="openshift-ingress/router-default-65cd7bbbb6-k5m7p" Sep 08 10:09:01.810276 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.810254 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-bound-sa-token\") pod \"image-registry-6d54b6c6c4-bxfp6\" (UID: \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\") " pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" Sep 08 10:09:01.818271 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.818259 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-524nf" Sep 08 10:09:01.829543 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.829441 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84lkj\" (UniqueName: \"kubernetes.io/projected/df6fbf45-0c71-4456-a95c-a5d5b24aa109-kube-api-access-84lkj\") pod \"cluster-samples-operator-7cdcfcf76c-qs5nv\" (UID: \"df6fbf45-0c71-4456-a95c-a5d5b24aa109\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qs5nv" Sep 08 10:09:01.906218 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:01.906189 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bjv7p" Sep 08 10:09:02.090023 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:02.090000 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-524nf"] Sep 08 10:09:02.092388 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:09:02.092351 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod852f4bd7_e2d8_4ca8_b37a_b16df9ec9a1a.slice/crio-4dc1d1f8d3efe4843a71c2a6046bc6e8c580ab4c338ba4c67eed6944d2664985 WatchSource:0}: Error finding container 4dc1d1f8d3efe4843a71c2a6046bc6e8c580ab4c338ba4c67eed6944d2664985: Status 404 returned error can't find the container with id 4dc1d1f8d3efe4843a71c2a6046bc6e8c580ab4c338ba4c67eed6944d2664985 Sep 08 10:09:02.173535 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:02.173516 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-bjv7p"] Sep 08 10:09:02.175178 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:09:02.175157 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1373d83d_e33c_4993_bfe5_7ff0b9241a79.slice/crio-fb9454b20a29478c6b78f1e2b0b1c4e727912101e3ca139ce9ec37fe1bd79b86 WatchSource:0}: Error finding container fb9454b20a29478c6b78f1e2b0b1c4e727912101e3ca139ce9ec37fe1bd79b86: Status 404 returned error can't find the container with id fb9454b20a29478c6b78f1e2b0b1c4e727912101e3ca139ce9ec37fe1bd79b86 Sep 08 10:09:02.261329 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:02.261309 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d28e6ab-62c9-45a3-ae14-18623958e9ef-service-ca-bundle\") pod \"router-default-65cd7bbbb6-k5m7p\" (UID: \"7d28e6ab-62c9-45a3-ae14-18623958e9ef\") " pod="openshift-ingress/router-default-65cd7bbbb6-k5m7p" Sep 08 10:09:02.261429 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:02.261340 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-registry-tls\") pod \"image-registry-6d54b6c6c4-bxfp6\" (UID: \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\") " pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" Sep 08 10:09:02.261429 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:02.261357 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d28e6ab-62c9-45a3-ae14-18623958e9ef-metrics-certs\") pod \"router-default-65cd7bbbb6-k5m7p\" (UID: \"7d28e6ab-62c9-45a3-ae14-18623958e9ef\") " pod="openshift-ingress/router-default-65cd7bbbb6-k5m7p" Sep 08 10:09:02.261520 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:02.261461 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 08 10:09:02.261520 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:02.261473 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d54b6c6c4-bxfp6: secret "image-registry-tls" not found Sep 08 10:09:02.261520 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:02.261471 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df6fbf45-0c71-4456-a95c-a5d5b24aa109-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-qs5nv\" (UID: \"df6fbf45-0c71-4456-a95c-a5d5b24aa109\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qs5nv" Sep 08 10:09:02.261520 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:02.261486 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7d28e6ab-62c9-45a3-ae14-18623958e9ef-service-ca-bundle podName:7d28e6ab-62c9-45a3-ae14-18623958e9ef nodeName:}" failed. No retries permitted until 2026-09-08 10:09:03.261470178 +0000 UTC m=+127.305574228 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7d28e6ab-62c9-45a3-ae14-18623958e9ef-service-ca-bundle") pod "router-default-65cd7bbbb6-k5m7p" (UID: "7d28e6ab-62c9-45a3-ae14-18623958e9ef") : configmap references non-existent config key: service-ca.crt Sep 08 10:09:02.261520 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:02.261516 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-registry-tls podName:4c12bdd1-d3f2-4659-be40-e653c94d7cbe nodeName:}" failed. No retries permitted until 2026-09-08 10:09:03.261506476 +0000 UTC m=+127.305610523 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-registry-tls") pod "image-registry-6d54b6c6c4-bxfp6" (UID: "4c12bdd1-d3f2-4659-be40-e653c94d7cbe") : secret "image-registry-tls" not found Sep 08 10:09:02.261520 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:02.261518 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 08 10:09:02.261747 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:02.261551 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d28e6ab-62c9-45a3-ae14-18623958e9ef-metrics-certs podName:7d28e6ab-62c9-45a3-ae14-18623958e9ef nodeName:}" failed. No retries permitted until 2026-09-08 10:09:03.261540276 +0000 UTC m=+127.305644322 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d28e6ab-62c9-45a3-ae14-18623958e9ef-metrics-certs") pod "router-default-65cd7bbbb6-k5m7p" (UID: "7d28e6ab-62c9-45a3-ae14-18623958e9ef") : secret "router-metrics-certs-default" not found Sep 08 10:09:02.261747 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:02.261567 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 08 10:09:02.261747 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:02.261619 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df6fbf45-0c71-4456-a95c-a5d5b24aa109-samples-operator-tls podName:df6fbf45-0c71-4456-a95c-a5d5b24aa109 nodeName:}" failed. No retries permitted until 2026-09-08 10:09:03.261602801 +0000 UTC m=+127.305706860 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/df6fbf45-0c71-4456-a95c-a5d5b24aa109-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-qs5nv" (UID: "df6fbf45-0c71-4456-a95c-a5d5b24aa109") : secret "samples-operator-tls" not found Sep 08 10:09:02.781660 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:02.781625 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bjv7p" event={"ID":"1373d83d-e33c-4993-bfe5-7ff0b9241a79","Type":"ContainerStarted","Data":"fb9454b20a29478c6b78f1e2b0b1c4e727912101e3ca139ce9ec37fe1bd79b86"} Sep 08 10:09:02.782609 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:02.782589 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-524nf" event={"ID":"852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a","Type":"ContainerStarted","Data":"4dc1d1f8d3efe4843a71c2a6046bc6e8c580ab4c338ba4c67eed6944d2664985"} Sep 08 10:09:03.268478 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:03.268442 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-registry-tls\") pod \"image-registry-6d54b6c6c4-bxfp6\" (UID: \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\") " pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" Sep 08 10:09:03.268478 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:03.268482 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d28e6ab-62c9-45a3-ae14-18623958e9ef-metrics-certs\") pod \"router-default-65cd7bbbb6-k5m7p\" (UID: \"7d28e6ab-62c9-45a3-ae14-18623958e9ef\") " pod="openshift-ingress/router-default-65cd7bbbb6-k5m7p" Sep 08 10:09:03.268658 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:03.268507 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df6fbf45-0c71-4456-a95c-a5d5b24aa109-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-qs5nv\" (UID: \"df6fbf45-0c71-4456-a95c-a5d5b24aa109\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qs5nv" Sep 08 10:09:03.268658 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:03.268565 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d28e6ab-62c9-45a3-ae14-18623958e9ef-service-ca-bundle\") pod \"router-default-65cd7bbbb6-k5m7p\" (UID: \"7d28e6ab-62c9-45a3-ae14-18623958e9ef\") " pod="openshift-ingress/router-default-65cd7bbbb6-k5m7p" Sep 08 10:09:03.268658 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:03.268583 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 08 10:09:03.268658 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:03.268592 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 08 10:09:03.268658 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:03.268600 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d54b6c6c4-bxfp6: secret "image-registry-tls" not found Sep 08 10:09:03.268658 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:03.268652 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-registry-tls podName:4c12bdd1-d3f2-4659-be40-e653c94d7cbe nodeName:}" failed. No retries permitted until 2026-09-08 10:09:05.268637586 +0000 UTC m=+129.312741631 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-registry-tls") pod "image-registry-6d54b6c6c4-bxfp6" (UID: "4c12bdd1-d3f2-4659-be40-e653c94d7cbe") : secret "image-registry-tls" not found Sep 08 10:09:03.268836 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:03.268666 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d28e6ab-62c9-45a3-ae14-18623958e9ef-metrics-certs podName:7d28e6ab-62c9-45a3-ae14-18623958e9ef nodeName:}" failed. No retries permitted until 2026-09-08 10:09:05.268658932 +0000 UTC m=+129.312762977 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d28e6ab-62c9-45a3-ae14-18623958e9ef-metrics-certs") pod "router-default-65cd7bbbb6-k5m7p" (UID: "7d28e6ab-62c9-45a3-ae14-18623958e9ef") : secret "router-metrics-certs-default" not found Sep 08 10:09:03.268836 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:03.268669 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 08 10:09:03.268836 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:03.268676 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7d28e6ab-62c9-45a3-ae14-18623958e9ef-service-ca-bundle podName:7d28e6ab-62c9-45a3-ae14-18623958e9ef nodeName:}" failed. No retries permitted until 2026-09-08 10:09:05.268670549 +0000 UTC m=+129.312774595 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7d28e6ab-62c9-45a3-ae14-18623958e9ef-service-ca-bundle") pod "router-default-65cd7bbbb6-k5m7p" (UID: "7d28e6ab-62c9-45a3-ae14-18623958e9ef") : configmap references non-existent config key: service-ca.crt Sep 08 10:09:03.268836 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:03.268736 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df6fbf45-0c71-4456-a95c-a5d5b24aa109-samples-operator-tls podName:df6fbf45-0c71-4456-a95c-a5d5b24aa109 nodeName:}" failed. No retries permitted until 2026-09-08 10:09:05.268703803 +0000 UTC m=+129.312807849 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/df6fbf45-0c71-4456-a95c-a5d5b24aa109-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-qs5nv" (UID: "df6fbf45-0c71-4456-a95c-a5d5b24aa109") : secret "samples-operator-tls" not found Sep 08 10:09:05.082843 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:05.082806 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b4e4397-1898-4201-9dcd-b5775ee1789c-metrics-certs\") pod \"network-metrics-daemon-mbs64\" (UID: \"3b4e4397-1898-4201-9dcd-b5775ee1789c\") " pod="openshift-multus/network-metrics-daemon-mbs64" Sep 08 10:09:05.083164 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:05.082961 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 08 10:09:05.083164 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:05.083023 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b4e4397-1898-4201-9dcd-b5775ee1789c-metrics-certs podName:3b4e4397-1898-4201-9dcd-b5775ee1789c nodeName:}" failed. No retries permitted until 2026-09-08 10:11:07.083006693 +0000 UTC m=+251.127110743 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b4e4397-1898-4201-9dcd-b5775ee1789c-metrics-certs") pod "network-metrics-daemon-mbs64" (UID: "3b4e4397-1898-4201-9dcd-b5775ee1789c") : secret "metrics-daemon-secret" not found Sep 08 10:09:05.284019 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:05.283948 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df6fbf45-0c71-4456-a95c-a5d5b24aa109-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-qs5nv\" (UID: \"df6fbf45-0c71-4456-a95c-a5d5b24aa109\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qs5nv" Sep 08 10:09:05.284019 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:05.284006 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d28e6ab-62c9-45a3-ae14-18623958e9ef-service-ca-bundle\") pod \"router-default-65cd7bbbb6-k5m7p\" (UID: \"7d28e6ab-62c9-45a3-ae14-18623958e9ef\") " pod="openshift-ingress/router-default-65cd7bbbb6-k5m7p" Sep 08 10:09:05.284210 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:05.284028 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-registry-tls\") pod \"image-registry-6d54b6c6c4-bxfp6\" (UID: \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\") " pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" Sep 08 10:09:05.284210 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:05.284044 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d28e6ab-62c9-45a3-ae14-18623958e9ef-metrics-certs\") pod \"router-default-65cd7bbbb6-k5m7p\" (UID: \"7d28e6ab-62c9-45a3-ae14-18623958e9ef\") " pod="openshift-ingress/router-default-65cd7bbbb6-k5m7p" Sep 08 10:09:05.284210 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:05.284115 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 08 10:09:05.284210 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:05.284127 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 08 10:09:05.284210 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:05.284136 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 08 10:09:05.284210 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:05.284143 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d54b6c6c4-bxfp6: secret "image-registry-tls" not found Sep 08 10:09:05.284210 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:05.284139 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7d28e6ab-62c9-45a3-ae14-18623958e9ef-service-ca-bundle podName:7d28e6ab-62c9-45a3-ae14-18623958e9ef nodeName:}" failed. No retries permitted until 2026-09-08 10:09:09.284119364 +0000 UTC m=+133.328223413 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7d28e6ab-62c9-45a3-ae14-18623958e9ef-service-ca-bundle") pod "router-default-65cd7bbbb6-k5m7p" (UID: "7d28e6ab-62c9-45a3-ae14-18623958e9ef") : configmap references non-existent config key: service-ca.crt Sep 08 10:09:05.284210 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:05.284179 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-registry-tls podName:4c12bdd1-d3f2-4659-be40-e653c94d7cbe nodeName:}" failed. No retries permitted until 2026-09-08 10:09:09.28416748 +0000 UTC m=+133.328271530 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-registry-tls") pod "image-registry-6d54b6c6c4-bxfp6" (UID: "4c12bdd1-d3f2-4659-be40-e653c94d7cbe") : secret "image-registry-tls" not found Sep 08 10:09:05.284210 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:05.284191 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df6fbf45-0c71-4456-a95c-a5d5b24aa109-samples-operator-tls podName:df6fbf45-0c71-4456-a95c-a5d5b24aa109 nodeName:}" failed. No retries permitted until 2026-09-08 10:09:09.284185474 +0000 UTC m=+133.328289519 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/df6fbf45-0c71-4456-a95c-a5d5b24aa109-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-qs5nv" (UID: "df6fbf45-0c71-4456-a95c-a5d5b24aa109") : secret "samples-operator-tls" not found Sep 08 10:09:05.284210 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:05.284202 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d28e6ab-62c9-45a3-ae14-18623958e9ef-metrics-certs podName:7d28e6ab-62c9-45a3-ae14-18623958e9ef nodeName:}" failed. No retries permitted until 2026-09-08 10:09:09.284196993 +0000 UTC m=+133.328301039 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d28e6ab-62c9-45a3-ae14-18623958e9ef-metrics-certs") pod "router-default-65cd7bbbb6-k5m7p" (UID: "7d28e6ab-62c9-45a3-ae14-18623958e9ef") : secret "router-metrics-certs-default" not found Sep 08 10:09:05.790597 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:05.790567 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-524nf" event={"ID":"852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a","Type":"ContainerStarted","Data":"19b2f751294924eedd1ef3cb52043dd6975f198583897fd9ebd8be459272e180"} Sep 08 10:09:05.815133 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:05.815089 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-524nf" podStartSLOduration=2.012263832 podStartE2EDuration="4.815075479s" totalImagesPullingTime="2.802811647s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:09:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:09:05.813615008 +0000 UTC m=+129.857719090" watchObservedRunningTime="2026-09-08 10:09:05.815075479 +0000 UTC m=+129.859179588" Sep 08 10:09:05.992458 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:05.992403 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-c5frh"] Sep 08 10:09:05.994733 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:05.994713 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-c5frh" Sep 08 10:09:05.998319 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:05.998288 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fpk9p\"" Sep 08 10:09:05.998656 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:05.998637 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 08 10:09:05.998756 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:05.998637 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 08 10:09:06.012667 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:06.012641 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-c5frh"] Sep 08 10:09:06.089758 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:06.089697 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c7957153-b146-457f-804c-55a93eb856a0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c5frh\" (UID: \"c7957153-b146-457f-804c-55a93eb856a0\") " pod="openshift-insights/insights-runtime-extractor-c5frh" Sep 08 10:09:06.089758 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:06.089739 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xjps\" (UniqueName: \"kubernetes.io/projected/c7957153-b146-457f-804c-55a93eb856a0-kube-api-access-2xjps\") pod \"insights-runtime-extractor-c5frh\" (UID: \"c7957153-b146-457f-804c-55a93eb856a0\") " pod="openshift-insights/insights-runtime-extractor-c5frh" Sep 08 10:09:06.090114 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:06.089769 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c7957153-b146-457f-804c-55a93eb856a0-data-volume\") pod \"insights-runtime-extractor-c5frh\" (UID: \"c7957153-b146-457f-804c-55a93eb856a0\") " pod="openshift-insights/insights-runtime-extractor-c5frh" Sep 08 10:09:06.090114 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:06.089899 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c7957153-b146-457f-804c-55a93eb856a0-crio-socket\") pod \"insights-runtime-extractor-c5frh\" (UID: \"c7957153-b146-457f-804c-55a93eb856a0\") " pod="openshift-insights/insights-runtime-extractor-c5frh" Sep 08 10:09:06.090114 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:06.089943 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c7957153-b146-457f-804c-55a93eb856a0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c5frh\" (UID: \"c7957153-b146-457f-804c-55a93eb856a0\") " pod="openshift-insights/insights-runtime-extractor-c5frh" Sep 08 10:09:06.190677 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:06.190650 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c7957153-b146-457f-804c-55a93eb856a0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c5frh\" (UID: \"c7957153-b146-457f-804c-55a93eb856a0\") " pod="openshift-insights/insights-runtime-extractor-c5frh" Sep 08 10:09:06.190812 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:06.190690 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xjps\" (UniqueName: \"kubernetes.io/projected/c7957153-b146-457f-804c-55a93eb856a0-kube-api-access-2xjps\") pod \"insights-runtime-extractor-c5frh\" (UID: \"c7957153-b146-457f-804c-55a93eb856a0\") " pod="openshift-insights/insights-runtime-extractor-c5frh" Sep 08 10:09:06.190812 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:06.190732 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c7957153-b146-457f-804c-55a93eb856a0-data-volume\") pod \"insights-runtime-extractor-c5frh\" (UID: \"c7957153-b146-457f-804c-55a93eb856a0\") " pod="openshift-insights/insights-runtime-extractor-c5frh" Sep 08 10:09:06.190812 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:06.190779 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 08 10:09:06.190812 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:06.190810 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c7957153-b146-457f-804c-55a93eb856a0-crio-socket\") pod \"insights-runtime-extractor-c5frh\" (UID: \"c7957153-b146-457f-804c-55a93eb856a0\") " pod="openshift-insights/insights-runtime-extractor-c5frh" Sep 08 10:09:06.191015 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:06.190846 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c7957153-b146-457f-804c-55a93eb856a0-kube-rbac-proxy-cm podName:c7957153-b146-457f-804c-55a93eb856a0 nodeName:}" failed. No retries permitted until 2026-09-08 10:09:06.690824214 +0000 UTC m=+130.734928274 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c7957153-b146-457f-804c-55a93eb856a0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-c5frh" (UID: "c7957153-b146-457f-804c-55a93eb856a0") : configmap "kube-rbac-proxy" not found Sep 08 10:09:06.191015 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:06.190872 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c7957153-b146-457f-804c-55a93eb856a0-crio-socket\") pod \"insights-runtime-extractor-c5frh\" (UID: \"c7957153-b146-457f-804c-55a93eb856a0\") " pod="openshift-insights/insights-runtime-extractor-c5frh" Sep 08 10:09:06.191015 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:06.190870 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c7957153-b146-457f-804c-55a93eb856a0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c5frh\" (UID: \"c7957153-b146-457f-804c-55a93eb856a0\") " pod="openshift-insights/insights-runtime-extractor-c5frh" Sep 08 10:09:06.191015 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:06.190948 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 08 10:09:06.191015 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:06.190995 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7957153-b146-457f-804c-55a93eb856a0-insights-runtime-extractor-tls podName:c7957153-b146-457f-804c-55a93eb856a0 nodeName:}" failed. No retries permitted until 2026-09-08 10:09:06.690982407 +0000 UTC m=+130.735086475 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c7957153-b146-457f-804c-55a93eb856a0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-c5frh" (UID: "c7957153-b146-457f-804c-55a93eb856a0") : secret "insights-runtime-extractor-tls" not found Sep 08 10:09:06.191252 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:06.191089 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c7957153-b146-457f-804c-55a93eb856a0-data-volume\") pod \"insights-runtime-extractor-c5frh\" (UID: \"c7957153-b146-457f-804c-55a93eb856a0\") " pod="openshift-insights/insights-runtime-extractor-c5frh" Sep 08 10:09:06.212817 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:06.212798 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xjps\" (UniqueName: \"kubernetes.io/projected/c7957153-b146-457f-804c-55a93eb856a0-kube-api-access-2xjps\") pod \"insights-runtime-extractor-c5frh\" (UID: \"c7957153-b146-457f-804c-55a93eb856a0\") " pod="openshift-insights/insights-runtime-extractor-c5frh" Sep 08 10:09:06.694711 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:06.694682 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c7957153-b146-457f-804c-55a93eb856a0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c5frh\" (UID: \"c7957153-b146-457f-804c-55a93eb856a0\") " pod="openshift-insights/insights-runtime-extractor-c5frh" Sep 08 10:09:06.694886 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:06.694728 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c7957153-b146-457f-804c-55a93eb856a0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c5frh\" (UID: \"c7957153-b146-457f-804c-55a93eb856a0\") " pod="openshift-insights/insights-runtime-extractor-c5frh" Sep 08 10:09:06.694886 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:06.694815 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 08 10:09:06.694886 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:06.694842 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 08 10:09:06.694886 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:06.694859 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c7957153-b146-457f-804c-55a93eb856a0-kube-rbac-proxy-cm podName:c7957153-b146-457f-804c-55a93eb856a0 nodeName:}" failed. No retries permitted until 2026-09-08 10:09:07.694845548 +0000 UTC m=+131.738949593 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c7957153-b146-457f-804c-55a93eb856a0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-c5frh" (UID: "c7957153-b146-457f-804c-55a93eb856a0") : configmap "kube-rbac-proxy" not found Sep 08 10:09:06.695036 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:06.694894 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7957153-b146-457f-804c-55a93eb856a0-insights-runtime-extractor-tls podName:c7957153-b146-457f-804c-55a93eb856a0 nodeName:}" failed. No retries permitted until 2026-09-08 10:09:07.694876838 +0000 UTC m=+131.738980883 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c7957153-b146-457f-804c-55a93eb856a0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-c5frh" (UID: "c7957153-b146-457f-804c-55a93eb856a0") : secret "insights-runtime-extractor-tls" not found Sep 08 10:09:07.702214 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:07.702130 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c7957153-b146-457f-804c-55a93eb856a0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c5frh\" (UID: \"c7957153-b146-457f-804c-55a93eb856a0\") " pod="openshift-insights/insights-runtime-extractor-c5frh" Sep 08 10:09:07.702214 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:07.702198 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c7957153-b146-457f-804c-55a93eb856a0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c5frh\" (UID: \"c7957153-b146-457f-804c-55a93eb856a0\") " pod="openshift-insights/insights-runtime-extractor-c5frh" Sep 08 10:09:07.702649 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:07.702280 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 08 10:09:07.702649 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:07.702340 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7957153-b146-457f-804c-55a93eb856a0-insights-runtime-extractor-tls podName:c7957153-b146-457f-804c-55a93eb856a0 nodeName:}" failed. No retries permitted until 2026-09-08 10:09:09.702325801 +0000 UTC m=+133.746429847 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c7957153-b146-457f-804c-55a93eb856a0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-c5frh" (UID: "c7957153-b146-457f-804c-55a93eb856a0") : secret "insights-runtime-extractor-tls" not found Sep 08 10:09:07.702649 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:07.702284 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 08 10:09:07.702649 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:07.702435 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c7957153-b146-457f-804c-55a93eb856a0-kube-rbac-proxy-cm podName:c7957153-b146-457f-804c-55a93eb856a0 nodeName:}" failed. No retries permitted until 2026-09-08 10:09:09.702404405 +0000 UTC m=+133.746508458 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c7957153-b146-457f-804c-55a93eb856a0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-c5frh" (UID: "c7957153-b146-457f-804c-55a93eb856a0") : configmap "kube-rbac-proxy" not found Sep 08 10:09:07.797590 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:07.797561 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bjv7p" event={"ID":"1373d83d-e33c-4993-bfe5-7ff0b9241a79","Type":"ContainerStarted","Data":"3bdf0f114aac8e1a7b586e499185b93839ad4d0da9eb0af533310d974e0cf0b8"} Sep 08 10:09:07.813850 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:07.813811 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bjv7p" podStartSLOduration=1.576892008 podStartE2EDuration="6.813792368s" totalImagesPullingTime="5.23690036s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:09:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:09:07.813394062 +0000 UTC m=+131.857498162" watchObservedRunningTime="2026-09-08 10:09:07.813792368 +0000 UTC m=+131.857896437" Sep 08 10:09:08.105447 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:08.105373 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vngnd_29e04309-88ba-49b2-aab0-4b68da8efd96/dns-node-resolver/0.log" Sep 08 10:09:08.712779 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:08.712754 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4gqq4_cfac6ed7-3115-4f50-9271-6a240f950ae5/node-ca/0.log" Sep 08 10:09:09.311615 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:09.311584 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df6fbf45-0c71-4456-a95c-a5d5b24aa109-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-qs5nv\" (UID: \"df6fbf45-0c71-4456-a95c-a5d5b24aa109\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qs5nv" Sep 08 10:09:09.311770 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:09.311651 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d28e6ab-62c9-45a3-ae14-18623958e9ef-service-ca-bundle\") pod \"router-default-65cd7bbbb6-k5m7p\" (UID: \"7d28e6ab-62c9-45a3-ae14-18623958e9ef\") " pod="openshift-ingress/router-default-65cd7bbbb6-k5m7p" Sep 08 10:09:09.311770 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:09.311674 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-registry-tls\") pod \"image-registry-6d54b6c6c4-bxfp6\" (UID: \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\") " pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" Sep 08 10:09:09.311770 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:09.311689 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d28e6ab-62c9-45a3-ae14-18623958e9ef-metrics-certs\") pod \"router-default-65cd7bbbb6-k5m7p\" (UID: \"7d28e6ab-62c9-45a3-ae14-18623958e9ef\") " pod="openshift-ingress/router-default-65cd7bbbb6-k5m7p" Sep 08 10:09:09.311770 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:09.311733 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 08 10:09:09.311770 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:09.311761 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 08 10:09:09.311988 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:09.311803 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7d28e6ab-62c9-45a3-ae14-18623958e9ef-service-ca-bundle podName:7d28e6ab-62c9-45a3-ae14-18623958e9ef nodeName:}" failed. No retries permitted until 2026-09-08 10:09:17.311784159 +0000 UTC m=+141.355888222 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7d28e6ab-62c9-45a3-ae14-18623958e9ef-service-ca-bundle") pod "router-default-65cd7bbbb6-k5m7p" (UID: "7d28e6ab-62c9-45a3-ae14-18623958e9ef") : configmap references non-existent config key: service-ca.crt Sep 08 10:09:09.311988 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:09.311820 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df6fbf45-0c71-4456-a95c-a5d5b24aa109-samples-operator-tls podName:df6fbf45-0c71-4456-a95c-a5d5b24aa109 nodeName:}" failed. No retries permitted until 2026-09-08 10:09:17.311813801 +0000 UTC m=+141.355917846 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/df6fbf45-0c71-4456-a95c-a5d5b24aa109-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-qs5nv" (UID: "df6fbf45-0c71-4456-a95c-a5d5b24aa109") : secret "samples-operator-tls" not found Sep 08 10:09:09.311988 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:09.311818 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 08 10:09:09.311988 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:09.311840 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d54b6c6c4-bxfp6: secret "image-registry-tls" not found Sep 08 10:09:09.311988 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:09.311830 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d28e6ab-62c9-45a3-ae14-18623958e9ef-metrics-certs podName:7d28e6ab-62c9-45a3-ae14-18623958e9ef nodeName:}" failed. No retries permitted until 2026-09-08 10:09:17.311824618 +0000 UTC m=+141.355928664 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d28e6ab-62c9-45a3-ae14-18623958e9ef-metrics-certs") pod "router-default-65cd7bbbb6-k5m7p" (UID: "7d28e6ab-62c9-45a3-ae14-18623958e9ef") : secret "router-metrics-certs-default" not found Sep 08 10:09:09.311988 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:09.311898 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-registry-tls podName:4c12bdd1-d3f2-4659-be40-e653c94d7cbe nodeName:}" failed. No retries permitted until 2026-09-08 10:09:17.311882306 +0000 UTC m=+141.355986365 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-registry-tls") pod "image-registry-6d54b6c6c4-bxfp6" (UID: "4c12bdd1-d3f2-4659-be40-e653c94d7cbe") : secret "image-registry-tls" not found Sep 08 10:09:09.714835 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:09.714804 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c7957153-b146-457f-804c-55a93eb856a0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c5frh\" (UID: \"c7957153-b146-457f-804c-55a93eb856a0\") " pod="openshift-insights/insights-runtime-extractor-c5frh" Sep 08 10:09:09.715189 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:09.714845 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c7957153-b146-457f-804c-55a93eb856a0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c5frh\" (UID: \"c7957153-b146-457f-804c-55a93eb856a0\") " pod="openshift-insights/insights-runtime-extractor-c5frh" Sep 08 10:09:09.715189 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:09.714914 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 08 10:09:09.715189 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:09.714957 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 08 10:09:09.715189 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:09.714964 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c7957153-b146-457f-804c-55a93eb856a0-kube-rbac-proxy-cm podName:c7957153-b146-457f-804c-55a93eb856a0 nodeName:}" failed. No retries permitted until 2026-09-08 10:09:13.71495104 +0000 UTC m=+137.759055086 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c7957153-b146-457f-804c-55a93eb856a0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-c5frh" (UID: "c7957153-b146-457f-804c-55a93eb856a0") : configmap "kube-rbac-proxy" not found Sep 08 10:09:09.715189 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:09.715023 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7957153-b146-457f-804c-55a93eb856a0-insights-runtime-extractor-tls podName:c7957153-b146-457f-804c-55a93eb856a0 nodeName:}" failed. No retries permitted until 2026-09-08 10:09:13.715002595 +0000 UTC m=+137.759106644 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c7957153-b146-457f-804c-55a93eb856a0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-c5frh" (UID: "c7957153-b146-457f-804c-55a93eb856a0") : secret "insights-runtime-extractor-tls" not found Sep 08 10:09:09.906834 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:09.906806 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rhxl4_7bfb2025-199a-4235-a283-7d555387fd2c/serve-healthcheck-canary/0.log" Sep 08 10:09:10.927180 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:10.927142 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-75d99c484-vbsh2"] Sep 08 10:09:10.930110 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:10.930094 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-vbsh2" Sep 08 10:09:10.937985 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:10.937964 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Sep 08 10:09:10.938143 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:10.938055 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Sep 08 10:09:10.938143 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:10.938122 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Sep 08 10:09:10.938336 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:10.938321 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Sep 08 10:09:10.939002 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:10.938986 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-nvfm7\"" Sep 08 10:09:10.956657 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:10.956622 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-vbsh2"] Sep 08 10:09:11.024941 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:11.024901 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt4nk\" (UniqueName: \"kubernetes.io/projected/619f368d-2b06-486a-b541-c856ccd6db57-kube-api-access-qt4nk\") pod \"service-ca-75d99c484-vbsh2\" (UID: \"619f368d-2b06-486a-b541-c856ccd6db57\") " pod="openshift-service-ca/service-ca-75d99c484-vbsh2" Sep 08 10:09:11.025134 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:11.024966 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/619f368d-2b06-486a-b541-c856ccd6db57-signing-cabundle\") pod \"service-ca-75d99c484-vbsh2\" (UID: \"619f368d-2b06-486a-b541-c856ccd6db57\") " pod="openshift-service-ca/service-ca-75d99c484-vbsh2" Sep 08 10:09:11.025134 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:11.025028 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/619f368d-2b06-486a-b541-c856ccd6db57-signing-key\") pod \"service-ca-75d99c484-vbsh2\" (UID: \"619f368d-2b06-486a-b541-c856ccd6db57\") " pod="openshift-service-ca/service-ca-75d99c484-vbsh2" Sep 08 10:09:11.126393 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:11.126341 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/619f368d-2b06-486a-b541-c856ccd6db57-signing-cabundle\") pod \"service-ca-75d99c484-vbsh2\" (UID: \"619f368d-2b06-486a-b541-c856ccd6db57\") " pod="openshift-service-ca/service-ca-75d99c484-vbsh2" Sep 08 10:09:11.126606 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:11.126434 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/619f368d-2b06-486a-b541-c856ccd6db57-signing-key\") pod \"service-ca-75d99c484-vbsh2\" (UID: \"619f368d-2b06-486a-b541-c856ccd6db57\") " pod="openshift-service-ca/service-ca-75d99c484-vbsh2" Sep 08 10:09:11.126606 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:11.126506 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt4nk\" (UniqueName: \"kubernetes.io/projected/619f368d-2b06-486a-b541-c856ccd6db57-kube-api-access-qt4nk\") pod \"service-ca-75d99c484-vbsh2\" (UID: \"619f368d-2b06-486a-b541-c856ccd6db57\") " pod="openshift-service-ca/service-ca-75d99c484-vbsh2" Sep 08 10:09:11.127040 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:11.127018 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/619f368d-2b06-486a-b541-c856ccd6db57-signing-cabundle\") pod \"service-ca-75d99c484-vbsh2\" (UID: \"619f368d-2b06-486a-b541-c856ccd6db57\") " pod="openshift-service-ca/service-ca-75d99c484-vbsh2" Sep 08 10:09:11.128972 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:11.128947 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/619f368d-2b06-486a-b541-c856ccd6db57-signing-key\") pod \"service-ca-75d99c484-vbsh2\" (UID: \"619f368d-2b06-486a-b541-c856ccd6db57\") " pod="openshift-service-ca/service-ca-75d99c484-vbsh2" Sep 08 10:09:11.145253 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:11.145230 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt4nk\" (UniqueName: \"kubernetes.io/projected/619f368d-2b06-486a-b541-c856ccd6db57-kube-api-access-qt4nk\") pod \"service-ca-75d99c484-vbsh2\" (UID: \"619f368d-2b06-486a-b541-c856ccd6db57\") " pod="openshift-service-ca/service-ca-75d99c484-vbsh2" Sep 08 10:09:11.238978 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:11.238875 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-vbsh2" Sep 08 10:09:11.513553 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:11.513443 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-vbsh2"] Sep 08 10:09:11.515935 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:09:11.515907 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod619f368d_2b06_486a_b541_c856ccd6db57.slice/crio-37770ce2b2b4fe1401c02962b173c9ff2b85e21f7861603b1257b5482ac1c1cf WatchSource:0}: Error finding container 37770ce2b2b4fe1401c02962b173c9ff2b85e21f7861603b1257b5482ac1c1cf: Status 404 returned error can't find the container with id 37770ce2b2b4fe1401c02962b173c9ff2b85e21f7861603b1257b5482ac1c1cf Sep 08 10:09:11.810546 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:11.810463 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-vbsh2" event={"ID":"619f368d-2b06-486a-b541-c856ccd6db57","Type":"ContainerStarted","Data":"a00aeeb6f503fc6b77706221f538e750d085f51ba104228dd496da5fc8ffca44"} Sep 08 10:09:11.810546 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:11.810509 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-vbsh2" event={"ID":"619f368d-2b06-486a-b541-c856ccd6db57","Type":"ContainerStarted","Data":"37770ce2b2b4fe1401c02962b173c9ff2b85e21f7861603b1257b5482ac1c1cf"} Sep 08 10:09:11.832234 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:11.832185 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-75d99c484-vbsh2" podStartSLOduration=1.8321689829999999 podStartE2EDuration="1.832168983s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:09:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:09:11.832161983 +0000 UTC m=+135.876266069" watchObservedRunningTime="2026-09-08 10:09:11.832168983 +0000 UTC m=+135.876273073" Sep 08 10:09:13.748388 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:13.748354 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c7957153-b146-457f-804c-55a93eb856a0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c5frh\" (UID: \"c7957153-b146-457f-804c-55a93eb856a0\") " pod="openshift-insights/insights-runtime-extractor-c5frh" Sep 08 10:09:13.748878 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:13.748468 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c7957153-b146-457f-804c-55a93eb856a0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c5frh\" (UID: \"c7957153-b146-457f-804c-55a93eb856a0\") " pod="openshift-insights/insights-runtime-extractor-c5frh" Sep 08 10:09:13.748878 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:13.748504 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 08 10:09:13.748878 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:13.748564 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 08 10:09:13.748878 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:13.748598 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c7957153-b146-457f-804c-55a93eb856a0-kube-rbac-proxy-cm podName:c7957153-b146-457f-804c-55a93eb856a0 nodeName:}" failed. No retries permitted until 2026-09-08 10:09:21.748575457 +0000 UTC m=+145.792679516 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c7957153-b146-457f-804c-55a93eb856a0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-c5frh" (UID: "c7957153-b146-457f-804c-55a93eb856a0") : configmap "kube-rbac-proxy" not found Sep 08 10:09:13.748878 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:13.748617 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7957153-b146-457f-804c-55a93eb856a0-insights-runtime-extractor-tls podName:c7957153-b146-457f-804c-55a93eb856a0 nodeName:}" failed. No retries permitted until 2026-09-08 10:09:21.748605189 +0000 UTC m=+145.792709235 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c7957153-b146-457f-804c-55a93eb856a0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-c5frh" (UID: "c7957153-b146-457f-804c-55a93eb856a0") : secret "insights-runtime-extractor-tls" not found Sep 08 10:09:17.377131 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:17.377098 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df6fbf45-0c71-4456-a95c-a5d5b24aa109-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-qs5nv\" (UID: \"df6fbf45-0c71-4456-a95c-a5d5b24aa109\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qs5nv" Sep 08 10:09:17.377525 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:17.377190 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d28e6ab-62c9-45a3-ae14-18623958e9ef-service-ca-bundle\") pod \"router-default-65cd7bbbb6-k5m7p\" (UID: \"7d28e6ab-62c9-45a3-ae14-18623958e9ef\") " pod="openshift-ingress/router-default-65cd7bbbb6-k5m7p" Sep 08 10:09:17.377525 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:17.377217 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-registry-tls\") pod \"image-registry-6d54b6c6c4-bxfp6\" (UID: \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\") " pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" Sep 08 10:09:17.377525 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:17.377233 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d28e6ab-62c9-45a3-ae14-18623958e9ef-metrics-certs\") pod \"router-default-65cd7bbbb6-k5m7p\" (UID: \"7d28e6ab-62c9-45a3-ae14-18623958e9ef\") " pod="openshift-ingress/router-default-65cd7bbbb6-k5m7p" Sep 08 10:09:17.377525 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:17.377332 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 08 10:09:17.377525 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:17.377382 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7d28e6ab-62c9-45a3-ae14-18623958e9ef-service-ca-bundle podName:7d28e6ab-62c9-45a3-ae14-18623958e9ef nodeName:}" failed. No retries permitted until 2026-09-08 10:09:33.377361435 +0000 UTC m=+157.421465554 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7d28e6ab-62c9-45a3-ae14-18623958e9ef-service-ca-bundle") pod "router-default-65cd7bbbb6-k5m7p" (UID: "7d28e6ab-62c9-45a3-ae14-18623958e9ef") : configmap references non-existent config key: service-ca.crt Sep 08 10:09:17.377525 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:17.377399 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d28e6ab-62c9-45a3-ae14-18623958e9ef-metrics-certs podName:7d28e6ab-62c9-45a3-ae14-18623958e9ef nodeName:}" failed. No retries permitted until 2026-09-08 10:09:33.377392177 +0000 UTC m=+157.421496223 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d28e6ab-62c9-45a3-ae14-18623958e9ef-metrics-certs") pod "router-default-65cd7bbbb6-k5m7p" (UID: "7d28e6ab-62c9-45a3-ae14-18623958e9ef") : secret "router-metrics-certs-default" not found Sep 08 10:09:17.379672 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:17.379643 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df6fbf45-0c71-4456-a95c-a5d5b24aa109-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-qs5nv\" (UID: \"df6fbf45-0c71-4456-a95c-a5d5b24aa109\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qs5nv" Sep 08 10:09:17.379770 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:17.379649 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-registry-tls\") pod \"image-registry-6d54b6c6c4-bxfp6\" (UID: \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\") " pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" Sep 08 10:09:17.521180 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:17.521156 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" Sep 08 10:09:17.523917 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:17.523899 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qs5nv" Sep 08 10:09:17.802045 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:17.802023 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d54b6c6c4-bxfp6"] Sep 08 10:09:17.804381 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:09:17.804350 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c12bdd1_d3f2_4659_be40_e653c94d7cbe.slice/crio-151840ac476e7082063e922a07f60b50e2e8ab439e144fcd501cc6da3069e1a8 WatchSource:0}: Error finding container 151840ac476e7082063e922a07f60b50e2e8ab439e144fcd501cc6da3069e1a8: Status 404 returned error can't find the container with id 151840ac476e7082063e922a07f60b50e2e8ab439e144fcd501cc6da3069e1a8 Sep 08 10:09:17.804967 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:17.804919 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qs5nv"] Sep 08 10:09:17.831310 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:17.831280 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" event={"ID":"4c12bdd1-d3f2-4659-be40-e653c94d7cbe","Type":"ContainerStarted","Data":"151840ac476e7082063e922a07f60b50e2e8ab439e144fcd501cc6da3069e1a8"} Sep 08 10:09:18.835636 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:18.835597 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qs5nv" event={"ID":"df6fbf45-0c71-4456-a95c-a5d5b24aa109","Type":"ContainerStarted","Data":"fe19cea0c3e4dd043d2c233ade11bd4c97a05acdb00f7d0cb185091c658b8141"} Sep 08 10:09:18.837006 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:18.836982 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" event={"ID":"4c12bdd1-d3f2-4659-be40-e653c94d7cbe","Type":"ContainerStarted","Data":"21a6e3f1723433946989592ebd401e8fdd8b8a42ce64a0aa199cf740ea6f2c6c"} Sep 08 10:09:18.837128 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:18.837111 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" Sep 08 10:09:18.863374 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:18.863325 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" podStartSLOduration=17.863309857 podStartE2EDuration="17.863309857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:09:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:09:18.863290433 +0000 UTC m=+142.907394529" watchObservedRunningTime="2026-09-08 10:09:18.863309857 +0000 UTC m=+142.907413928" Sep 08 10:09:21.810741 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:21.810710 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c7957153-b146-457f-804c-55a93eb856a0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c5frh\" (UID: \"c7957153-b146-457f-804c-55a93eb856a0\") " pod="openshift-insights/insights-runtime-extractor-c5frh" Sep 08 10:09:21.811119 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:21.810750 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c7957153-b146-457f-804c-55a93eb856a0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c5frh\" (UID: \"c7957153-b146-457f-804c-55a93eb856a0\") " pod="openshift-insights/insights-runtime-extractor-c5frh" Sep 08 10:09:21.811119 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:21.810854 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 08 10:09:21.811119 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:21.810916 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7957153-b146-457f-804c-55a93eb856a0-insights-runtime-extractor-tls podName:c7957153-b146-457f-804c-55a93eb856a0 nodeName:}" failed. No retries permitted until 2026-09-08 10:09:37.810899916 +0000 UTC m=+161.855003963 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c7957153-b146-457f-804c-55a93eb856a0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-c5frh" (UID: "c7957153-b146-457f-804c-55a93eb856a0") : secret "insights-runtime-extractor-tls" not found Sep 08 10:09:21.811270 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:21.811174 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c7957153-b146-457f-804c-55a93eb856a0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c5frh\" (UID: \"c7957153-b146-457f-804c-55a93eb856a0\") " pod="openshift-insights/insights-runtime-extractor-c5frh" Sep 08 10:09:21.849687 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:21.849660 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qs5nv" event={"ID":"df6fbf45-0c71-4456-a95c-a5d5b24aa109","Type":"ContainerStarted","Data":"9150c6433b957a40855f88b045125d5d0300a93ea4d0d2ff0cbb3085b2eeac44"} Sep 08 10:09:21.849814 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:21.849726 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qs5nv" event={"ID":"df6fbf45-0c71-4456-a95c-a5d5b24aa109","Type":"ContainerStarted","Data":"98b1957eb1e6d10010f5b3582f4827d7ec389f44bbd81a5509488c0f6135e440"} Sep 08 10:09:21.867776 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:21.867734 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qs5nv" podStartSLOduration=17.647838217 podStartE2EDuration="20.867721659s" totalImagesPullingTime="3.219883442s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:09:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:09:21.866997513 +0000 UTC m=+145.911101601" watchObservedRunningTime="2026-09-08 10:09:21.867721659 +0000 UTC m=+145.911825726" Sep 08 10:09:32.827009 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:32.826966 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-dl46k" podUID="0045c129-e95a-4d80-a1b6-c972ee592d76" Sep 08 10:09:32.880768 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:32.880742 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dl46k" Sep 08 10:09:33.396186 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:33.396153 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d28e6ab-62c9-45a3-ae14-18623958e9ef-service-ca-bundle\") pod \"router-default-65cd7bbbb6-k5m7p\" (UID: \"7d28e6ab-62c9-45a3-ae14-18623958e9ef\") " pod="openshift-ingress/router-default-65cd7bbbb6-k5m7p" Sep 08 10:09:33.396341 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:33.396194 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d28e6ab-62c9-45a3-ae14-18623958e9ef-metrics-certs\") pod \"router-default-65cd7bbbb6-k5m7p\" (UID: \"7d28e6ab-62c9-45a3-ae14-18623958e9ef\") " pod="openshift-ingress/router-default-65cd7bbbb6-k5m7p" Sep 08 10:09:33.396756 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:33.396740 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d28e6ab-62c9-45a3-ae14-18623958e9ef-service-ca-bundle\") pod \"router-default-65cd7bbbb6-k5m7p\" (UID: \"7d28e6ab-62c9-45a3-ae14-18623958e9ef\") " pod="openshift-ingress/router-default-65cd7bbbb6-k5m7p" Sep 08 10:09:33.398552 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:33.398531 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d28e6ab-62c9-45a3-ae14-18623958e9ef-metrics-certs\") pod \"router-default-65cd7bbbb6-k5m7p\" (UID: \"7d28e6ab-62c9-45a3-ae14-18623958e9ef\") " pod="openshift-ingress/router-default-65cd7bbbb6-k5m7p" Sep 08 10:09:33.413626 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:33.413433 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-65cd7bbbb6-k5m7p" Sep 08 10:09:33.694692 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:33.694667 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-65cd7bbbb6-k5m7p"] Sep 08 10:09:33.697005 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:09:33.696980 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d28e6ab_62c9_45a3_ae14_18623958e9ef.slice/crio-370d3b5bad32156698cc176823484f7a3406c4c7d34282bacb7cb4a6992e660b WatchSource:0}: Error finding container 370d3b5bad32156698cc176823484f7a3406c4c7d34282bacb7cb4a6992e660b: Status 404 returned error can't find the container with id 370d3b5bad32156698cc176823484f7a3406c4c7d34282bacb7cb4a6992e660b Sep 08 10:09:33.885530 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:33.885497 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-65cd7bbbb6-k5m7p" event={"ID":"7d28e6ab-62c9-45a3-ae14-18623958e9ef","Type":"ContainerStarted","Data":"7df8abe0dd207e1de1ab38d43b9e633f0e13417e1c0b1410b808cd22696dd3bf"} Sep 08 10:09:33.885901 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:33.885570 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-65cd7bbbb6-k5m7p" event={"ID":"7d28e6ab-62c9-45a3-ae14-18623958e9ef","Type":"ContainerStarted","Data":"370d3b5bad32156698cc176823484f7a3406c4c7d34282bacb7cb4a6992e660b"} Sep 08 10:09:33.904752 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:33.904706 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-65cd7bbbb6-k5m7p" podStartSLOduration=32.904691844 podStartE2EDuration="32.904691844s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:09:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:09:33.903335989 +0000 UTC m=+157.947440078" watchObservedRunningTime="2026-09-08 10:09:33.904691844 +0000 UTC m=+157.948795912" Sep 08 10:09:34.414582 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:34.414555 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-65cd7bbbb6-k5m7p" Sep 08 10:09:34.416774 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:34.416755 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-65cd7bbbb6-k5m7p" Sep 08 10:09:34.423551 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:34.423531 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-mbs64" podUID="3b4e4397-1898-4201-9dcd-b5775ee1789c" Sep 08 10:09:34.888874 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:34.888848 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-65cd7bbbb6-k5m7p" Sep 08 10:09:34.889969 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:34.889951 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-65cd7bbbb6-k5m7p" Sep 08 10:09:36.539349 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.539318 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-f22s7"] Sep 08 10:09:36.541254 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.541239 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-f22s7" Sep 08 10:09:36.544871 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.544856 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-tmvp2\"" Sep 08 10:09:36.545818 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.545803 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Sep 08 10:09:36.556083 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.556059 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-bw9mh"] Sep 08 10:09:36.557725 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.557710 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-57c9bfb9ff-fpg4s"] Sep 08 10:09:36.557857 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.557842 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-bw9mh" Sep 08 10:09:36.559493 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.559476 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57c9bfb9ff-fpg4s" Sep 08 10:09:36.561299 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.561275 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 08 10:09:36.561569 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.561555 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 08 10:09:36.562280 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.562268 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-tcv89\"" Sep 08 10:09:36.568514 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.568485 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6d54b6c6c4-bxfp6"] Sep 08 10:09:36.571387 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.571102 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-f22s7"] Sep 08 10:09:36.573812 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.573785 2568 patch_prober.go:28] interesting pod/image-registry-6d54b6c6c4-bxfp6 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 08 10:09:36.573911 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.573833 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" podUID="4c12bdd1-d3f2-4659-be40-e653c94d7cbe" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 08 10:09:36.577370 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.577353 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 08 10:09:36.577492 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.577477 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 08 10:09:36.577787 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.577775 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 08 10:09:36.577883 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.577867 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-sfcqc\"" Sep 08 10:09:36.578098 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.578085 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 08 10:09:36.582307 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.582292 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 08 10:09:36.594944 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.594923 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57c9bfb9ff-fpg4s"] Sep 08 10:09:36.605279 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.605249 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-bw9mh"] Sep 08 10:09:36.617238 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.617219 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2ec9be4b-99b8-495a-a8d8-8c010727c301-console-config\") pod \"console-57c9bfb9ff-fpg4s\" (UID: \"2ec9be4b-99b8-495a-a8d8-8c010727c301\") " pod="openshift-console/console-57c9bfb9ff-fpg4s" Sep 08 10:09:36.617322 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.617247 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vpc7\" (UniqueName: \"kubernetes.io/projected/2ec9be4b-99b8-495a-a8d8-8c010727c301-kube-api-access-4vpc7\") pod \"console-57c9bfb9ff-fpg4s\" (UID: \"2ec9be4b-99b8-495a-a8d8-8c010727c301\") " pod="openshift-console/console-57c9bfb9ff-fpg4s" Sep 08 10:09:36.617322 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.617274 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2ec9be4b-99b8-495a-a8d8-8c010727c301-service-ca\") pod \"console-57c9bfb9ff-fpg4s\" (UID: \"2ec9be4b-99b8-495a-a8d8-8c010727c301\") " pod="openshift-console/console-57c9bfb9ff-fpg4s" Sep 08 10:09:36.617322 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.617290 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74fqj\" (UniqueName: \"kubernetes.io/projected/b969204c-f958-48d9-9e4e-9827ed64496b-kube-api-access-74fqj\") pod \"downloads-85d98c998f-bw9mh\" (UID: \"b969204c-f958-48d9-9e4e-9827ed64496b\") " pod="openshift-console/downloads-85d98c998f-bw9mh" Sep 08 10:09:36.617322 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.617314 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2ec9be4b-99b8-495a-a8d8-8c010727c301-console-oauth-config\") pod \"console-57c9bfb9ff-fpg4s\" (UID: \"2ec9be4b-99b8-495a-a8d8-8c010727c301\") " pod="openshift-console/console-57c9bfb9ff-fpg4s" Sep 08 10:09:36.617469 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.617337 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a7659944-4498-4a41-b2d3-e58f182c80e1-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-f22s7\" (UID: \"a7659944-4498-4a41-b2d3-e58f182c80e1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-f22s7" Sep 08 10:09:36.617469 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.617356 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2ec9be4b-99b8-495a-a8d8-8c010727c301-console-serving-cert\") pod \"console-57c9bfb9ff-fpg4s\" (UID: \"2ec9be4b-99b8-495a-a8d8-8c010727c301\") " pod="openshift-console/console-57c9bfb9ff-fpg4s" Sep 08 10:09:36.617531 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.617472 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2ec9be4b-99b8-495a-a8d8-8c010727c301-oauth-serving-cert\") pod \"console-57c9bfb9ff-fpg4s\" (UID: \"2ec9be4b-99b8-495a-a8d8-8c010727c301\") " pod="openshift-console/console-57c9bfb9ff-fpg4s" Sep 08 10:09:36.718457 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.718429 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2ec9be4b-99b8-495a-a8d8-8c010727c301-console-serving-cert\") pod \"console-57c9bfb9ff-fpg4s\" (UID: \"2ec9be4b-99b8-495a-a8d8-8c010727c301\") " pod="openshift-console/console-57c9bfb9ff-fpg4s" Sep 08 10:09:36.718571 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.718510 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2ec9be4b-99b8-495a-a8d8-8c010727c301-oauth-serving-cert\") pod \"console-57c9bfb9ff-fpg4s\" (UID: \"2ec9be4b-99b8-495a-a8d8-8c010727c301\") " pod="openshift-console/console-57c9bfb9ff-fpg4s" Sep 08 10:09:36.718571 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.718548 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2ec9be4b-99b8-495a-a8d8-8c010727c301-console-config\") pod \"console-57c9bfb9ff-fpg4s\" (UID: \"2ec9be4b-99b8-495a-a8d8-8c010727c301\") " pod="openshift-console/console-57c9bfb9ff-fpg4s" Sep 08 10:09:36.718691 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.718575 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vpc7\" (UniqueName: \"kubernetes.io/projected/2ec9be4b-99b8-495a-a8d8-8c010727c301-kube-api-access-4vpc7\") pod \"console-57c9bfb9ff-fpg4s\" (UID: \"2ec9be4b-99b8-495a-a8d8-8c010727c301\") " pod="openshift-console/console-57c9bfb9ff-fpg4s" Sep 08 10:09:36.718691 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.718613 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2ec9be4b-99b8-495a-a8d8-8c010727c301-service-ca\") pod \"console-57c9bfb9ff-fpg4s\" (UID: \"2ec9be4b-99b8-495a-a8d8-8c010727c301\") " pod="openshift-console/console-57c9bfb9ff-fpg4s" Sep 08 10:09:36.718691 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.718640 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74fqj\" (UniqueName: \"kubernetes.io/projected/b969204c-f958-48d9-9e4e-9827ed64496b-kube-api-access-74fqj\") pod \"downloads-85d98c998f-bw9mh\" (UID: \"b969204c-f958-48d9-9e4e-9827ed64496b\") " pod="openshift-console/downloads-85d98c998f-bw9mh" Sep 08 10:09:36.718691 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.718672 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2ec9be4b-99b8-495a-a8d8-8c010727c301-console-oauth-config\") pod \"console-57c9bfb9ff-fpg4s\" (UID: \"2ec9be4b-99b8-495a-a8d8-8c010727c301\") " pod="openshift-console/console-57c9bfb9ff-fpg4s" Sep 08 10:09:36.718889 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.718732 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a7659944-4498-4a41-b2d3-e58f182c80e1-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-f22s7\" (UID: \"a7659944-4498-4a41-b2d3-e58f182c80e1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-f22s7" Sep 08 10:09:36.719289 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.719264 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2ec9be4b-99b8-495a-a8d8-8c010727c301-oauth-serving-cert\") pod \"console-57c9bfb9ff-fpg4s\" (UID: \"2ec9be4b-99b8-495a-a8d8-8c010727c301\") " pod="openshift-console/console-57c9bfb9ff-fpg4s" Sep 08 10:09:36.719387 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.719332 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2ec9be4b-99b8-495a-a8d8-8c010727c301-console-config\") pod \"console-57c9bfb9ff-fpg4s\" (UID: \"2ec9be4b-99b8-495a-a8d8-8c010727c301\") " pod="openshift-console/console-57c9bfb9ff-fpg4s" Sep 08 10:09:36.719387 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.719366 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2ec9be4b-99b8-495a-a8d8-8c010727c301-service-ca\") pod \"console-57c9bfb9ff-fpg4s\" (UID: \"2ec9be4b-99b8-495a-a8d8-8c010727c301\") " pod="openshift-console/console-57c9bfb9ff-fpg4s" Sep 08 10:09:36.721046 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.721018 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2ec9be4b-99b8-495a-a8d8-8c010727c301-console-serving-cert\") pod \"console-57c9bfb9ff-fpg4s\" (UID: \"2ec9be4b-99b8-495a-a8d8-8c010727c301\") " pod="openshift-console/console-57c9bfb9ff-fpg4s" Sep 08 10:09:36.721178 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.721161 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a7659944-4498-4a41-b2d3-e58f182c80e1-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-f22s7\" (UID: \"a7659944-4498-4a41-b2d3-e58f182c80e1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-f22s7" Sep 08 10:09:36.721216 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.721202 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2ec9be4b-99b8-495a-a8d8-8c010727c301-console-oauth-config\") pod \"console-57c9bfb9ff-fpg4s\" (UID: \"2ec9be4b-99b8-495a-a8d8-8c010727c301\") " pod="openshift-console/console-57c9bfb9ff-fpg4s" Sep 08 10:09:36.735591 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.735561 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vpc7\" (UniqueName: \"kubernetes.io/projected/2ec9be4b-99b8-495a-a8d8-8c010727c301-kube-api-access-4vpc7\") pod \"console-57c9bfb9ff-fpg4s\" (UID: \"2ec9be4b-99b8-495a-a8d8-8c010727c301\") " pod="openshift-console/console-57c9bfb9ff-fpg4s" Sep 08 10:09:36.736944 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.736922 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74fqj\" (UniqueName: \"kubernetes.io/projected/b969204c-f958-48d9-9e4e-9827ed64496b-kube-api-access-74fqj\") pod \"downloads-85d98c998f-bw9mh\" (UID: \"b969204c-f958-48d9-9e4e-9827ed64496b\") " pod="openshift-console/downloads-85d98c998f-bw9mh" Sep 08 10:09:36.849973 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.849916 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-f22s7" Sep 08 10:09:36.868065 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.868046 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-bw9mh" Sep 08 10:09:36.874718 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:36.874698 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57c9bfb9ff-fpg4s" Sep 08 10:09:37.132998 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:37.132853 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-f22s7"] Sep 08 10:09:37.135381 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:09:37.135356 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7659944_4498_4a41_b2d3_e58f182c80e1.slice/crio-c8b07c1dcbb8102bbd8dd285de786872ad808487ac4261e492abfccc5565263a WatchSource:0}: Error finding container c8b07c1dcbb8102bbd8dd285de786872ad808487ac4261e492abfccc5565263a: Status 404 returned error can't find the container with id c8b07c1dcbb8102bbd8dd285de786872ad808487ac4261e492abfccc5565263a Sep 08 10:09:37.157286 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:37.157266 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-bw9mh"] Sep 08 10:09:37.160697 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:09:37.160674 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb969204c_f958_48d9_9e4e_9827ed64496b.slice/crio-877cb10a22551e68d3bb664ff7e65e5d70ee63dce32f628366229be288c785fb WatchSource:0}: Error finding container 877cb10a22551e68d3bb664ff7e65e5d70ee63dce32f628366229be288c785fb: Status 404 returned error can't find the container with id 877cb10a22551e68d3bb664ff7e65e5d70ee63dce32f628366229be288c785fb Sep 08 10:09:37.178001 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:37.177979 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57c9bfb9ff-fpg4s"] Sep 08 10:09:37.180224 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:09:37.180206 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ec9be4b_99b8_495a_a8d8_8c010727c301.slice/crio-3a9f162354cf00621dee66d52e39975aca5581790b1bf4588b0de6c3d7181234 WatchSource:0}: Error finding container 3a9f162354cf00621dee66d52e39975aca5581790b1bf4588b0de6c3d7181234: Status 404 returned error can't find the container with id 3a9f162354cf00621dee66d52e39975aca5581790b1bf4588b0de6c3d7181234 Sep 08 10:09:37.728225 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:37.728183 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0045c129-e95a-4d80-a1b6-c972ee592d76-metrics-tls\") pod \"dns-default-dl46k\" (UID: \"0045c129-e95a-4d80-a1b6-c972ee592d76\") " pod="openshift-dns/dns-default-dl46k" Sep 08 10:09:37.731094 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:37.731069 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0045c129-e95a-4d80-a1b6-c972ee592d76-metrics-tls\") pod \"dns-default-dl46k\" (UID: \"0045c129-e95a-4d80-a1b6-c972ee592d76\") " pod="openshift-dns/dns-default-dl46k" Sep 08 10:09:37.829648 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:37.829603 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c7957153-b146-457f-804c-55a93eb856a0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c5frh\" (UID: \"c7957153-b146-457f-804c-55a93eb856a0\") " pod="openshift-insights/insights-runtime-extractor-c5frh" Sep 08 10:09:37.832330 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:37.832302 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c7957153-b146-457f-804c-55a93eb856a0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c5frh\" (UID: \"c7957153-b146-457f-804c-55a93eb856a0\") " pod="openshift-insights/insights-runtime-extractor-c5frh" Sep 08 10:09:37.901927 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:37.901880 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57c9bfb9ff-fpg4s" event={"ID":"2ec9be4b-99b8-495a-a8d8-8c010727c301","Type":"ContainerStarted","Data":"3a9f162354cf00621dee66d52e39975aca5581790b1bf4588b0de6c3d7181234"} Sep 08 10:09:37.903317 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:37.903278 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-bw9mh" event={"ID":"b969204c-f958-48d9-9e4e-9827ed64496b","Type":"ContainerStarted","Data":"877cb10a22551e68d3bb664ff7e65e5d70ee63dce32f628366229be288c785fb"} Sep 08 10:09:37.904894 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:37.904838 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-f22s7" event={"ID":"a7659944-4498-4a41-b2d3-e58f182c80e1","Type":"ContainerStarted","Data":"c8b07c1dcbb8102bbd8dd285de786872ad808487ac4261e492abfccc5565263a"} Sep 08 10:09:37.984868 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:37.984788 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-t8hl6\"" Sep 08 10:09:37.992874 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:37.992849 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dl46k" Sep 08 10:09:38.105167 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:38.105047 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-c5frh" Sep 08 10:09:38.282765 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:38.282736 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dl46k"] Sep 08 10:09:38.310034 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:09:38.310000 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0045c129_e95a_4d80_a1b6_c972ee592d76.slice/crio-70f065c45e96173d925498a6c37902c63291ecbfe47372a3cf0a88bc17a67c88 WatchSource:0}: Error finding container 70f065c45e96173d925498a6c37902c63291ecbfe47372a3cf0a88bc17a67c88: Status 404 returned error can't find the container with id 70f065c45e96173d925498a6c37902c63291ecbfe47372a3cf0a88bc17a67c88 Sep 08 10:09:38.597105 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:38.597025 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-c5frh"] Sep 08 10:09:38.600888 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:09:38.600858 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7957153_b146_457f_804c_55a93eb856a0.slice/crio-9459a224f814b62f9d8cbcb6276fd4ecc6dfcffba585d6934ee037369973be0a WatchSource:0}: Error finding container 9459a224f814b62f9d8cbcb6276fd4ecc6dfcffba585d6934ee037369973be0a: Status 404 returned error can't find the container with id 9459a224f814b62f9d8cbcb6276fd4ecc6dfcffba585d6934ee037369973be0a Sep 08 10:09:38.911436 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:38.911364 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-c5frh" event={"ID":"c7957153-b146-457f-804c-55a93eb856a0","Type":"ContainerStarted","Data":"1b07ea7bd8eb7124895cbf2e92b947f3e1f370de7929cc1ab98ea5233e8a4591"} Sep 08 10:09:38.911895 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:38.911455 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-c5frh" event={"ID":"c7957153-b146-457f-804c-55a93eb856a0","Type":"ContainerStarted","Data":"9459a224f814b62f9d8cbcb6276fd4ecc6dfcffba585d6934ee037369973be0a"} Sep 08 10:09:38.913910 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:38.913878 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-f22s7" event={"ID":"a7659944-4498-4a41-b2d3-e58f182c80e1","Type":"ContainerStarted","Data":"a9522370d479eb3e664af212b3e95b7dec3e1c4bc9ca007423a137abe282b13f"} Sep 08 10:09:38.915112 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:38.915052 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-f22s7" Sep 08 10:09:38.916636 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:38.916576 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dl46k" event={"ID":"0045c129-e95a-4d80-a1b6-c972ee592d76","Type":"ContainerStarted","Data":"70f065c45e96173d925498a6c37902c63291ecbfe47372a3cf0a88bc17a67c88"} Sep 08 10:09:38.921571 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:38.921549 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-f22s7" Sep 08 10:09:38.941566 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:38.941432 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-f22s7" podStartSLOduration=1.744589524 podStartE2EDuration="2.941398323s" totalImagesPullingTime="1.196808799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:09:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:09:38.939596624 +0000 UTC m=+162.983700720" watchObservedRunningTime="2026-09-08 10:09:38.941398323 +0000 UTC m=+162.985502447" Sep 08 10:09:39.423900 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:39.422477 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-lvjmv"] Sep 08 10:09:39.426757 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:39.425304 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-lvjmv" Sep 08 10:09:39.434097 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:39.432573 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 08 10:09:39.434097 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:39.432855 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 08 10:09:39.434097 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:39.433102 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 08 10:09:39.434097 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:39.433265 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 08 10:09:39.434097 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:39.433444 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-dr6l6\"" Sep 08 10:09:39.436507 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:39.436485 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 08 10:09:39.443571 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:39.443287 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-lvjmv"] Sep 08 10:09:39.444453 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:39.444374 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c58ff7e6-5d97-430e-95ae-7a86cfd47680-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-lvjmv\" (UID: \"c58ff7e6-5d97-430e-95ae-7a86cfd47680\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-lvjmv" Sep 08 10:09:39.444567 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:39.444456 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nctz\" (UniqueName: \"kubernetes.io/projected/c58ff7e6-5d97-430e-95ae-7a86cfd47680-kube-api-access-5nctz\") pod \"prometheus-operator-6f4946db56-lvjmv\" (UID: \"c58ff7e6-5d97-430e-95ae-7a86cfd47680\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-lvjmv" Sep 08 10:09:39.444567 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:39.444496 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c58ff7e6-5d97-430e-95ae-7a86cfd47680-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-lvjmv\" (UID: \"c58ff7e6-5d97-430e-95ae-7a86cfd47680\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-lvjmv" Sep 08 10:09:39.445117 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:39.444673 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c58ff7e6-5d97-430e-95ae-7a86cfd47680-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-lvjmv\" (UID: \"c58ff7e6-5d97-430e-95ae-7a86cfd47680\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-lvjmv" Sep 08 10:09:39.545831 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:39.545774 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nctz\" (UniqueName: \"kubernetes.io/projected/c58ff7e6-5d97-430e-95ae-7a86cfd47680-kube-api-access-5nctz\") pod \"prometheus-operator-6f4946db56-lvjmv\" (UID: \"c58ff7e6-5d97-430e-95ae-7a86cfd47680\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-lvjmv" Sep 08 10:09:39.545831 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:39.545825 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c58ff7e6-5d97-430e-95ae-7a86cfd47680-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-lvjmv\" (UID: \"c58ff7e6-5d97-430e-95ae-7a86cfd47680\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-lvjmv" Sep 08 10:09:39.546073 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:39.545921 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c58ff7e6-5d97-430e-95ae-7a86cfd47680-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-lvjmv\" (UID: \"c58ff7e6-5d97-430e-95ae-7a86cfd47680\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-lvjmv" Sep 08 10:09:39.546073 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:39.545962 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c58ff7e6-5d97-430e-95ae-7a86cfd47680-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-lvjmv\" (UID: \"c58ff7e6-5d97-430e-95ae-7a86cfd47680\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-lvjmv" Sep 08 10:09:39.546659 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:39.546338 2568 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Sep 08 10:09:39.546659 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:39.546436 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c58ff7e6-5d97-430e-95ae-7a86cfd47680-prometheus-operator-tls podName:c58ff7e6-5d97-430e-95ae-7a86cfd47680 nodeName:}" failed. No retries permitted until 2026-09-08 10:09:40.046393595 +0000 UTC m=+164.090497657 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/c58ff7e6-5d97-430e-95ae-7a86cfd47680-prometheus-operator-tls") pod "prometheus-operator-6f4946db56-lvjmv" (UID: "c58ff7e6-5d97-430e-95ae-7a86cfd47680") : secret "prometheus-operator-tls" not found Sep 08 10:09:39.546659 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:39.546625 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c58ff7e6-5d97-430e-95ae-7a86cfd47680-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-lvjmv\" (UID: \"c58ff7e6-5d97-430e-95ae-7a86cfd47680\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-lvjmv" Sep 08 10:09:39.550840 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:39.549572 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c58ff7e6-5d97-430e-95ae-7a86cfd47680-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-lvjmv\" (UID: \"c58ff7e6-5d97-430e-95ae-7a86cfd47680\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-lvjmv" Sep 08 10:09:39.555580 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:39.555535 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nctz\" (UniqueName: \"kubernetes.io/projected/c58ff7e6-5d97-430e-95ae-7a86cfd47680-kube-api-access-5nctz\") pod \"prometheus-operator-6f4946db56-lvjmv\" (UID: \"c58ff7e6-5d97-430e-95ae-7a86cfd47680\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-lvjmv" Sep 08 10:09:40.049814 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:40.049650 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c58ff7e6-5d97-430e-95ae-7a86cfd47680-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-lvjmv\" (UID: \"c58ff7e6-5d97-430e-95ae-7a86cfd47680\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-lvjmv" Sep 08 10:09:40.052815 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:40.052767 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c58ff7e6-5d97-430e-95ae-7a86cfd47680-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-lvjmv\" (UID: \"c58ff7e6-5d97-430e-95ae-7a86cfd47680\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-lvjmv" Sep 08 10:09:40.343270 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:40.343214 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-lvjmv" Sep 08 10:09:40.643119 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:40.643038 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-lvjmv"] Sep 08 10:09:40.928232 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:40.928147 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dl46k" event={"ID":"0045c129-e95a-4d80-a1b6-c972ee592d76","Type":"ContainerStarted","Data":"159d6ade43c5805a9abc63ea11884a5995e1807eb6269150ecf4157888693178"} Sep 08 10:09:41.715883 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:09:41.715558 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc58ff7e6_5d97_430e_95ae_7a86cfd47680.slice/crio-d1ec1235b2cefd4c5bf657a15bdab39ce1fa6efd2fd63ea8b624198d6ae886a4 WatchSource:0}: Error finding container d1ec1235b2cefd4c5bf657a15bdab39ce1fa6efd2fd63ea8b624198d6ae886a4: Status 404 returned error can't find the container with id d1ec1235b2cefd4c5bf657a15bdab39ce1fa6efd2fd63ea8b624198d6ae886a4 Sep 08 10:09:41.934543 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:41.934504 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dl46k" event={"ID":"0045c129-e95a-4d80-a1b6-c972ee592d76","Type":"ContainerStarted","Data":"6f8db9f8991ba14cbe2975c51bebcc8be88d3f592047d0b074072e67d7e3bd08"} Sep 08 10:09:41.934688 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:41.934671 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dl46k" Sep 08 10:09:41.935959 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:41.935930 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-lvjmv" event={"ID":"c58ff7e6-5d97-430e-95ae-7a86cfd47680","Type":"ContainerStarted","Data":"d1ec1235b2cefd4c5bf657a15bdab39ce1fa6efd2fd63ea8b624198d6ae886a4"} Sep 08 10:09:41.938023 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:41.938002 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-c5frh" event={"ID":"c7957153-b146-457f-804c-55a93eb856a0","Type":"ContainerStarted","Data":"4d753c037b6488a6a92919d48de709a367b48d15d2341dac1cb102a9d8b2a203"} Sep 08 10:09:41.939779 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:41.939755 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57c9bfb9ff-fpg4s" event={"ID":"2ec9be4b-99b8-495a-a8d8-8c010727c301","Type":"ContainerStarted","Data":"67f3cfaf81306b7ec3ff565b71b824ac25ea987f6a0d806fd3f56b602490c59a"} Sep 08 10:09:41.956377 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:41.956338 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-dl46k" podStartSLOduration=131.032116522 podStartE2EDuration="2m12.95632498s" totalImagesPullingTime="1.924208458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:07:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:09:41.955913961 +0000 UTC m=+166.000018051" watchObservedRunningTime="2026-09-08 10:09:41.95632498 +0000 UTC m=+166.000429050" Sep 08 10:09:41.976481 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:41.976135 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-57c9bfb9ff-fpg4s" podStartSLOduration=1.414798255 podStartE2EDuration="5.97611644s" totalImagesPullingTime="4.561318185s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:09:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:09:41.975485754 +0000 UTC m=+166.019589843" watchObservedRunningTime="2026-09-08 10:09:41.97611644 +0000 UTC m=+166.020220532" Sep 08 10:09:43.951515 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:43.951472 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-lvjmv" event={"ID":"c58ff7e6-5d97-430e-95ae-7a86cfd47680","Type":"ContainerStarted","Data":"278f00ab9764c1a29964217c4cac6d74e5dd1bd206fa9457c6966909ad4c322d"} Sep 08 10:09:43.951937 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:43.951549 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-lvjmv" event={"ID":"c58ff7e6-5d97-430e-95ae-7a86cfd47680","Type":"ContainerStarted","Data":"a1abfa8decca9276726a2845592448e21970f1427ed2ebc9df2e4894f6a383fb"} Sep 08 10:09:43.954206 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:43.954178 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-c5frh" event={"ID":"c7957153-b146-457f-804c-55a93eb856a0","Type":"ContainerStarted","Data":"4240a1e1c0584feefc25cd814b5b71da505fd76f7a601861594bd8f1313ff5b0"} Sep 08 10:09:43.970886 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:43.970834 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-lvjmv" podStartSLOduration=3.1970407339999998 podStartE2EDuration="4.970816229s" totalImagesPullingTime="1.773775495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:09:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:09:43.96882189 +0000 UTC m=+168.012925983" watchObservedRunningTime="2026-09-08 10:09:43.970816229 +0000 UTC m=+168.014920297" Sep 08 10:09:43.989536 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:43.989485 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-c5frh" podStartSLOduration=34.25675013 podStartE2EDuration="38.989465935s" totalImagesPullingTime="4.732715805s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 10:09:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:09:43.988011511 +0000 UTC m=+168.032115601" watchObservedRunningTime="2026-09-08 10:09:43.989465935 +0000 UTC m=+168.033570004" Sep 08 10:09:45.937076 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:45.936992 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-zbkhn"] Sep 08 10:09:45.941273 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:45.941251 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zbkhn" Sep 08 10:09:45.943268 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:45.943244 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-t589v"] Sep 08 10:09:45.943863 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:45.943840 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-gjcc7\"" Sep 08 10:09:45.944302 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:45.944247 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 08 10:09:45.944302 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:45.944261 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 08 10:09:45.945820 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:45.945331 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-t589v" Sep 08 10:09:45.951035 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:45.950906 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 08 10:09:45.951035 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:45.950980 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-hz64k\"" Sep 08 10:09:45.951179 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:45.951113 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 08 10:09:45.952059 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:45.952040 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 08 10:09:45.971292 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:45.971228 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-zbkhn"] Sep 08 10:09:46.005542 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.005371 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/042a44d9-65fb-4808-bced-291b70524783-node-exporter-wtmp\") pod \"node-exporter-t589v\" (UID: \"042a44d9-65fb-4808-bced-291b70524783\") " pod="openshift-monitoring/node-exporter-t589v" Sep 08 10:09:46.005542 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.005430 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/64726051-51dc-4318-a02f-edbc884e3654-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-zbkhn\" (UID: \"64726051-51dc-4318-a02f-edbc884e3654\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zbkhn" Sep 08 10:09:46.005542 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.005463 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/042a44d9-65fb-4808-bced-291b70524783-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t589v\" (UID: \"042a44d9-65fb-4808-bced-291b70524783\") " pod="openshift-monitoring/node-exporter-t589v" Sep 08 10:09:46.005542 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.005496 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqr6d\" (UniqueName: \"kubernetes.io/projected/64726051-51dc-4318-a02f-edbc884e3654-kube-api-access-dqr6d\") pod \"openshift-state-metrics-657fd6c54-zbkhn\" (UID: \"64726051-51dc-4318-a02f-edbc884e3654\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zbkhn" Sep 08 10:09:46.005838 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.005554 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64726051-51dc-4318-a02f-edbc884e3654-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-zbkhn\" (UID: \"64726051-51dc-4318-a02f-edbc884e3654\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zbkhn" Sep 08 10:09:46.005838 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.005579 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/042a44d9-65fb-4808-bced-291b70524783-metrics-client-ca\") pod \"node-exporter-t589v\" (UID: \"042a44d9-65fb-4808-bced-291b70524783\") " pod="openshift-monitoring/node-exporter-t589v" Sep 08 10:09:46.005838 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.005602 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/64726051-51dc-4318-a02f-edbc884e3654-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-zbkhn\" (UID: \"64726051-51dc-4318-a02f-edbc884e3654\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zbkhn" Sep 08 10:09:46.005838 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.005622 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/042a44d9-65fb-4808-bced-291b70524783-sys\") pod \"node-exporter-t589v\" (UID: \"042a44d9-65fb-4808-bced-291b70524783\") " pod="openshift-monitoring/node-exporter-t589v" Sep 08 10:09:46.005838 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.005651 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/042a44d9-65fb-4808-bced-291b70524783-node-exporter-textfile\") pod \"node-exporter-t589v\" (UID: \"042a44d9-65fb-4808-bced-291b70524783\") " pod="openshift-monitoring/node-exporter-t589v" Sep 08 10:09:46.005838 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.005688 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/042a44d9-65fb-4808-bced-291b70524783-root\") pod \"node-exporter-t589v\" (UID: \"042a44d9-65fb-4808-bced-291b70524783\") " pod="openshift-monitoring/node-exporter-t589v" Sep 08 10:09:46.005838 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.005712 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lb948\" (UniqueName: \"kubernetes.io/projected/042a44d9-65fb-4808-bced-291b70524783-kube-api-access-lb948\") pod \"node-exporter-t589v\" (UID: \"042a44d9-65fb-4808-bced-291b70524783\") " pod="openshift-monitoring/node-exporter-t589v" Sep 08 10:09:46.005838 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.005739 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/042a44d9-65fb-4808-bced-291b70524783-node-exporter-tls\") pod \"node-exporter-t589v\" (UID: \"042a44d9-65fb-4808-bced-291b70524783\") " pod="openshift-monitoring/node-exporter-t589v" Sep 08 10:09:46.005838 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.005792 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/042a44d9-65fb-4808-bced-291b70524783-node-exporter-accelerators-collector-config\") pod \"node-exporter-t589v\" (UID: \"042a44d9-65fb-4808-bced-291b70524783\") " pod="openshift-monitoring/node-exporter-t589v" Sep 08 10:09:46.107191 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.107161 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/042a44d9-65fb-4808-bced-291b70524783-node-exporter-textfile\") pod \"node-exporter-t589v\" (UID: \"042a44d9-65fb-4808-bced-291b70524783\") " pod="openshift-monitoring/node-exporter-t589v" Sep 08 10:09:46.107331 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.107211 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/042a44d9-65fb-4808-bced-291b70524783-root\") pod \"node-exporter-t589v\" (UID: \"042a44d9-65fb-4808-bced-291b70524783\") " pod="openshift-monitoring/node-exporter-t589v" Sep 08 10:09:46.107331 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.107237 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lb948\" (UniqueName: \"kubernetes.io/projected/042a44d9-65fb-4808-bced-291b70524783-kube-api-access-lb948\") pod \"node-exporter-t589v\" (UID: \"042a44d9-65fb-4808-bced-291b70524783\") " pod="openshift-monitoring/node-exporter-t589v" Sep 08 10:09:46.107331 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.107265 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/042a44d9-65fb-4808-bced-291b70524783-node-exporter-tls\") pod \"node-exporter-t589v\" (UID: \"042a44d9-65fb-4808-bced-291b70524783\") " pod="openshift-monitoring/node-exporter-t589v" Sep 08 10:09:46.107331 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.107295 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/042a44d9-65fb-4808-bced-291b70524783-root\") pod \"node-exporter-t589v\" (UID: \"042a44d9-65fb-4808-bced-291b70524783\") " pod="openshift-monitoring/node-exporter-t589v" Sep 08 10:09:46.107331 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.107310 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/042a44d9-65fb-4808-bced-291b70524783-node-exporter-accelerators-collector-config\") pod \"node-exporter-t589v\" (UID: \"042a44d9-65fb-4808-bced-291b70524783\") " pod="openshift-monitoring/node-exporter-t589v" Sep 08 10:09:46.107672 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.107372 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/042a44d9-65fb-4808-bced-291b70524783-node-exporter-wtmp\") pod \"node-exporter-t589v\" (UID: \"042a44d9-65fb-4808-bced-291b70524783\") " pod="openshift-monitoring/node-exporter-t589v" Sep 08 10:09:46.107672 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.107403 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/64726051-51dc-4318-a02f-edbc884e3654-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-zbkhn\" (UID: \"64726051-51dc-4318-a02f-edbc884e3654\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zbkhn" Sep 08 10:09:46.107672 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.107456 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/042a44d9-65fb-4808-bced-291b70524783-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t589v\" (UID: \"042a44d9-65fb-4808-bced-291b70524783\") " pod="openshift-monitoring/node-exporter-t589v" Sep 08 10:09:46.107672 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.107488 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqr6d\" (UniqueName: \"kubernetes.io/projected/64726051-51dc-4318-a02f-edbc884e3654-kube-api-access-dqr6d\") pod \"openshift-state-metrics-657fd6c54-zbkhn\" (UID: \"64726051-51dc-4318-a02f-edbc884e3654\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zbkhn" Sep 08 10:09:46.107672 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.107526 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64726051-51dc-4318-a02f-edbc884e3654-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-zbkhn\" (UID: \"64726051-51dc-4318-a02f-edbc884e3654\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zbkhn" Sep 08 10:09:46.107672 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.107554 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/042a44d9-65fb-4808-bced-291b70524783-metrics-client-ca\") pod \"node-exporter-t589v\" (UID: \"042a44d9-65fb-4808-bced-291b70524783\") " pod="openshift-monitoring/node-exporter-t589v" Sep 08 10:09:46.107672 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.107580 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/64726051-51dc-4318-a02f-edbc884e3654-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-zbkhn\" (UID: \"64726051-51dc-4318-a02f-edbc884e3654\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zbkhn" Sep 08 10:09:46.107672 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.107600 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/042a44d9-65fb-4808-bced-291b70524783-node-exporter-textfile\") pod \"node-exporter-t589v\" (UID: \"042a44d9-65fb-4808-bced-291b70524783\") " pod="openshift-monitoring/node-exporter-t589v" Sep 08 10:09:46.107672 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.107657 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/042a44d9-65fb-4808-bced-291b70524783-sys\") pod \"node-exporter-t589v\" (UID: \"042a44d9-65fb-4808-bced-291b70524783\") " pod="openshift-monitoring/node-exporter-t589v" Sep 08 10:09:46.107672 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.107606 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/042a44d9-65fb-4808-bced-291b70524783-sys\") pod \"node-exporter-t589v\" (UID: \"042a44d9-65fb-4808-bced-291b70524783\") " pod="openshift-monitoring/node-exporter-t589v" Sep 08 10:09:46.108158 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.107726 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/042a44d9-65fb-4808-bced-291b70524783-node-exporter-wtmp\") pod \"node-exporter-t589v\" (UID: \"042a44d9-65fb-4808-bced-291b70524783\") " pod="openshift-monitoring/node-exporter-t589v" Sep 08 10:09:46.108158 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.107842 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/042a44d9-65fb-4808-bced-291b70524783-node-exporter-accelerators-collector-config\") pod \"node-exporter-t589v\" (UID: \"042a44d9-65fb-4808-bced-291b70524783\") " pod="openshift-monitoring/node-exporter-t589v" Sep 08 10:09:46.108438 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.108367 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/042a44d9-65fb-4808-bced-291b70524783-metrics-client-ca\") pod \"node-exporter-t589v\" (UID: \"042a44d9-65fb-4808-bced-291b70524783\") " pod="openshift-monitoring/node-exporter-t589v" Sep 08 10:09:46.108438 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.108375 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64726051-51dc-4318-a02f-edbc884e3654-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-zbkhn\" (UID: \"64726051-51dc-4318-a02f-edbc884e3654\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zbkhn" Sep 08 10:09:46.110729 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.110703 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/042a44d9-65fb-4808-bced-291b70524783-node-exporter-tls\") pod \"node-exporter-t589v\" (UID: \"042a44d9-65fb-4808-bced-291b70524783\") " pod="openshift-monitoring/node-exporter-t589v" Sep 08 10:09:46.110729 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.110719 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/042a44d9-65fb-4808-bced-291b70524783-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t589v\" (UID: \"042a44d9-65fb-4808-bced-291b70524783\") " pod="openshift-monitoring/node-exporter-t589v" Sep 08 10:09:46.110884 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.110865 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/64726051-51dc-4318-a02f-edbc884e3654-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-zbkhn\" (UID: \"64726051-51dc-4318-a02f-edbc884e3654\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zbkhn" Sep 08 10:09:46.110940 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.110901 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/64726051-51dc-4318-a02f-edbc884e3654-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-zbkhn\" (UID: \"64726051-51dc-4318-a02f-edbc884e3654\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zbkhn" Sep 08 10:09:46.118328 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.118307 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqr6d\" (UniqueName: \"kubernetes.io/projected/64726051-51dc-4318-a02f-edbc884e3654-kube-api-access-dqr6d\") pod \"openshift-state-metrics-657fd6c54-zbkhn\" (UID: \"64726051-51dc-4318-a02f-edbc884e3654\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zbkhn" Sep 08 10:09:46.121849 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.121829 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lb948\" (UniqueName: \"kubernetes.io/projected/042a44d9-65fb-4808-bced-291b70524783-kube-api-access-lb948\") pod \"node-exporter-t589v\" (UID: \"042a44d9-65fb-4808-bced-291b70524783\") " pod="openshift-monitoring/node-exporter-t589v" Sep 08 10:09:46.255280 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.255206 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zbkhn" Sep 08 10:09:46.261312 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.261267 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-t589v" Sep 08 10:09:46.546132 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.546056 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-zbkhn"] Sep 08 10:09:46.550116 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:09:46.550089 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64726051_51dc_4318_a02f_edbc884e3654.slice/crio-2b787098791b8873adcedbb8f08b276d32a21d4aad481a09d45ff526967b4c17 WatchSource:0}: Error finding container 2b787098791b8873adcedbb8f08b276d32a21d4aad481a09d45ff526967b4c17: Status 404 returned error can't find the container with id 2b787098791b8873adcedbb8f08b276d32a21d4aad481a09d45ff526967b4c17 Sep 08 10:09:46.574233 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.574213 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" Sep 08 10:09:46.876001 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.875319 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-57c9bfb9ff-fpg4s" Sep 08 10:09:46.876765 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.876268 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-57c9bfb9ff-fpg4s" Sep 08 10:09:46.882620 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.882597 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-57c9bfb9ff-fpg4s" Sep 08 10:09:46.888968 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.887588 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 10:09:46.891263 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.891241 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:46.894703 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.894596 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 08 10:09:46.895125 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.894842 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 08 10:09:46.895125 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.894922 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 08 10:09:46.895125 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.895030 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 08 10:09:46.895743 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.895725 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 08 10:09:46.895951 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.895936 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 08 10:09:46.896431 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.896126 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-q629x\"" Sep 08 10:09:46.896431 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.896151 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 08 10:09:46.896431 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.896160 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 08 10:09:46.896431 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.896152 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 08 10:09:46.915057 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.914989 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 10:09:46.968163 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.968078 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t589v" event={"ID":"042a44d9-65fb-4808-bced-291b70524783","Type":"ContainerStarted","Data":"2495f793436f6a905bf477a4f1ce1e5f35e594b72b25a96f02a28611bf27a9cf"} Sep 08 10:09:46.970992 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.970463 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zbkhn" event={"ID":"64726051-51dc-4318-a02f-edbc884e3654","Type":"ContainerStarted","Data":"7082f293f75d59b80857ca0513060edcda23b6d5fc4ecc1cf3676e85aa2a0fe5"} Sep 08 10:09:46.970992 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.970532 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zbkhn" event={"ID":"64726051-51dc-4318-a02f-edbc884e3654","Type":"ContainerStarted","Data":"ddafdc6921299b893be0f05d2ccd24214f37d5a065317323cf122b93eb71d53a"} Sep 08 10:09:46.970992 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.970564 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zbkhn" event={"ID":"64726051-51dc-4318-a02f-edbc884e3654","Type":"ContainerStarted","Data":"2b787098791b8873adcedbb8f08b276d32a21d4aad481a09d45ff526967b4c17"} Sep 08 10:09:46.976208 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:46.976187 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-57c9bfb9ff-fpg4s" Sep 08 10:09:47.020597 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.019945 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/afcc6060-56ad-4294-8d71-c535213cecd2-config-out\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.020597 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.019982 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/afcc6060-56ad-4294-8d71-c535213cecd2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.020597 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.020009 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/afcc6060-56ad-4294-8d71-c535213cecd2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.020597 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.020058 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/afcc6060-56ad-4294-8d71-c535213cecd2-web-config\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.020597 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.020092 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8cq2\" (UniqueName: \"kubernetes.io/projected/afcc6060-56ad-4294-8d71-c535213cecd2-kube-api-access-c8cq2\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.020597 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.020127 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/afcc6060-56ad-4294-8d71-c535213cecd2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.020597 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.020156 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/afcc6060-56ad-4294-8d71-c535213cecd2-config-volume\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.020597 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.020217 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/afcc6060-56ad-4294-8d71-c535213cecd2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.020597 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.020268 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/afcc6060-56ad-4294-8d71-c535213cecd2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.020597 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.020300 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afcc6060-56ad-4294-8d71-c535213cecd2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.021136 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.020714 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/afcc6060-56ad-4294-8d71-c535213cecd2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.021136 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.020760 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/afcc6060-56ad-4294-8d71-c535213cecd2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.021136 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.020785 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/afcc6060-56ad-4294-8d71-c535213cecd2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.121573 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.121537 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8cq2\" (UniqueName: \"kubernetes.io/projected/afcc6060-56ad-4294-8d71-c535213cecd2-kube-api-access-c8cq2\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.121740 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.121589 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/afcc6060-56ad-4294-8d71-c535213cecd2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.121740 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.121619 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/afcc6060-56ad-4294-8d71-c535213cecd2-config-volume\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.121740 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.121656 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/afcc6060-56ad-4294-8d71-c535213cecd2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.121740 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.121695 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/afcc6060-56ad-4294-8d71-c535213cecd2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.121740 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.121725 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afcc6060-56ad-4294-8d71-c535213cecd2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.121994 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.121772 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/afcc6060-56ad-4294-8d71-c535213cecd2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.121994 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.121799 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/afcc6060-56ad-4294-8d71-c535213cecd2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.121994 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.121824 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/afcc6060-56ad-4294-8d71-c535213cecd2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.121994 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.121873 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/afcc6060-56ad-4294-8d71-c535213cecd2-config-out\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.121994 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.121895 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/afcc6060-56ad-4294-8d71-c535213cecd2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.121994 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.121918 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/afcc6060-56ad-4294-8d71-c535213cecd2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.121994 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.121947 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/afcc6060-56ad-4294-8d71-c535213cecd2-web-config\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.122327 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:47.122169 2568 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Sep 08 10:09:47.122327 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:09:47.122237 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afcc6060-56ad-4294-8d71-c535213cecd2-secret-alertmanager-main-tls podName:afcc6060-56ad-4294-8d71-c535213cecd2 nodeName:}" failed. No retries permitted until 2026-09-08 10:09:47.62221741 +0000 UTC m=+171.666321457 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/afcc6060-56ad-4294-8d71-c535213cecd2-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "afcc6060-56ad-4294-8d71-c535213cecd2") : secret "alertmanager-main-tls" not found Sep 08 10:09:47.122481 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.122458 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/afcc6060-56ad-4294-8d71-c535213cecd2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.122670 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.122651 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/afcc6060-56ad-4294-8d71-c535213cecd2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.124061 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.123961 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afcc6060-56ad-4294-8d71-c535213cecd2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.127807 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.127747 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/afcc6060-56ad-4294-8d71-c535213cecd2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.127807 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.127777 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/afcc6060-56ad-4294-8d71-c535213cecd2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.128598 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.128176 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/afcc6060-56ad-4294-8d71-c535213cecd2-config-out\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.128598 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.128182 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/afcc6060-56ad-4294-8d71-c535213cecd2-web-config\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.128598 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.128466 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/afcc6060-56ad-4294-8d71-c535213cecd2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.128598 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.128486 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/afcc6060-56ad-4294-8d71-c535213cecd2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.128598 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.128562 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/afcc6060-56ad-4294-8d71-c535213cecd2-config-volume\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.129607 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.129583 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/afcc6060-56ad-4294-8d71-c535213cecd2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.134811 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.134787 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8cq2\" (UniqueName: \"kubernetes.io/projected/afcc6060-56ad-4294-8d71-c535213cecd2-kube-api-access-c8cq2\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.627061 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.626924 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/afcc6060-56ad-4294-8d71-c535213cecd2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.630138 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.630109 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/afcc6060-56ad-4294-8d71-c535213cecd2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"afcc6060-56ad-4294-8d71-c535213cecd2\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:47.805377 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:47.805343 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:09:48.399115 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:48.399082 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbs64" Sep 08 10:09:50.816657 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:50.816624 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57c9bfb9ff-fpg4s"] Sep 08 10:09:51.948148 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:51.948110 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dl46k" Sep 08 10:09:54.907909 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:54.907800 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 10:09:54.911071 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:09:54.911046 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafcc6060_56ad_4294_8d71_c535213cecd2.slice/crio-1540f278a0818531cafe28b914e033bf29e2bc79bc4247506bcfb5209df4ce1f WatchSource:0}: Error finding container 1540f278a0818531cafe28b914e033bf29e2bc79bc4247506bcfb5209df4ce1f: Status 404 returned error can't find the container with id 1540f278a0818531cafe28b914e033bf29e2bc79bc4247506bcfb5209df4ce1f Sep 08 10:09:55.008951 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:55.008882 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-bw9mh" event={"ID":"b969204c-f958-48d9-9e4e-9827ed64496b","Type":"ContainerStarted","Data":"307320f972fb818c413a28bbb9a48206bc70382dfbef412a91048b262c7e2882"} Sep 08 10:09:55.009432 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:55.009280 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-bw9mh" Sep 08 10:09:55.012736 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:55.012648 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"afcc6060-56ad-4294-8d71-c535213cecd2","Type":"ContainerStarted","Data":"1540f278a0818531cafe28b914e033bf29e2bc79bc4247506bcfb5209df4ce1f"} Sep 08 10:09:55.015681 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:55.015241 2568 generic.go:356] "Generic (PLEG): container finished" podID="042a44d9-65fb-4808-bced-291b70524783" containerID="fe75132beb0670c34c9dc2de7d1e8c345a4bdf55106dd22425679e50c6d99028" exitCode=0 Sep 08 10:09:55.015681 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:55.015298 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t589v" event={"ID":"042a44d9-65fb-4808-bced-291b70524783","Type":"ContainerDied","Data":"fe75132beb0670c34c9dc2de7d1e8c345a4bdf55106dd22425679e50c6d99028"} Sep 08 10:09:55.020614 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:55.020594 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-bw9mh" Sep 08 10:09:55.030516 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:55.030469 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-bw9mh" podStartSLOduration=1.511015337 podStartE2EDuration="19.030452534s" totalImagesPullingTime="17.519437197s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:09:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:09:55.028214174 +0000 UTC m=+179.072318270" watchObservedRunningTime="2026-09-08 10:09:55.030452534 +0000 UTC m=+179.074556615" Sep 08 10:09:55.607688 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:55.607653 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-d4d69bc5c-nxgx2"] Sep 08 10:09:55.611577 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:55.611514 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d4d69bc5c-nxgx2" Sep 08 10:09:55.618802 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:55.618775 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 08 10:09:55.623136 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:55.623109 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d4d69bc5c-nxgx2"] Sep 08 10:09:55.706816 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:55.706790 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qxmd\" (UniqueName: \"kubernetes.io/projected/15b0b656-6513-415b-b04a-dddcb2be52ac-kube-api-access-2qxmd\") pod \"console-d4d69bc5c-nxgx2\" (UID: \"15b0b656-6513-415b-b04a-dddcb2be52ac\") " pod="openshift-console/console-d4d69bc5c-nxgx2" Sep 08 10:09:55.706983 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:55.706862 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15b0b656-6513-415b-b04a-dddcb2be52ac-console-oauth-config\") pod \"console-d4d69bc5c-nxgx2\" (UID: \"15b0b656-6513-415b-b04a-dddcb2be52ac\") " pod="openshift-console/console-d4d69bc5c-nxgx2" Sep 08 10:09:55.706983 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:55.706889 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15b0b656-6513-415b-b04a-dddcb2be52ac-console-config\") pod \"console-d4d69bc5c-nxgx2\" (UID: \"15b0b656-6513-415b-b04a-dddcb2be52ac\") " pod="openshift-console/console-d4d69bc5c-nxgx2" Sep 08 10:09:55.706983 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:55.706926 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15b0b656-6513-415b-b04a-dddcb2be52ac-trusted-ca-bundle\") pod \"console-d4d69bc5c-nxgx2\" (UID: \"15b0b656-6513-415b-b04a-dddcb2be52ac\") " pod="openshift-console/console-d4d69bc5c-nxgx2" Sep 08 10:09:55.706983 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:55.706960 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15b0b656-6513-415b-b04a-dddcb2be52ac-console-serving-cert\") pod \"console-d4d69bc5c-nxgx2\" (UID: \"15b0b656-6513-415b-b04a-dddcb2be52ac\") " pod="openshift-console/console-d4d69bc5c-nxgx2" Sep 08 10:09:55.707163 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:55.707060 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15b0b656-6513-415b-b04a-dddcb2be52ac-oauth-serving-cert\") pod \"console-d4d69bc5c-nxgx2\" (UID: \"15b0b656-6513-415b-b04a-dddcb2be52ac\") " pod="openshift-console/console-d4d69bc5c-nxgx2" Sep 08 10:09:55.707163 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:55.707092 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15b0b656-6513-415b-b04a-dddcb2be52ac-service-ca\") pod \"console-d4d69bc5c-nxgx2\" (UID: \"15b0b656-6513-415b-b04a-dddcb2be52ac\") " pod="openshift-console/console-d4d69bc5c-nxgx2" Sep 08 10:09:55.808036 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:55.808001 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15b0b656-6513-415b-b04a-dddcb2be52ac-console-oauth-config\") pod \"console-d4d69bc5c-nxgx2\" (UID: \"15b0b656-6513-415b-b04a-dddcb2be52ac\") " pod="openshift-console/console-d4d69bc5c-nxgx2" Sep 08 10:09:55.808216 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:55.808043 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15b0b656-6513-415b-b04a-dddcb2be52ac-console-config\") pod \"console-d4d69bc5c-nxgx2\" (UID: \"15b0b656-6513-415b-b04a-dddcb2be52ac\") " pod="openshift-console/console-d4d69bc5c-nxgx2" Sep 08 10:09:55.808216 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:55.808176 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15b0b656-6513-415b-b04a-dddcb2be52ac-trusted-ca-bundle\") pod \"console-d4d69bc5c-nxgx2\" (UID: \"15b0b656-6513-415b-b04a-dddcb2be52ac\") " pod="openshift-console/console-d4d69bc5c-nxgx2" Sep 08 10:09:55.808332 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:55.808217 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15b0b656-6513-415b-b04a-dddcb2be52ac-console-serving-cert\") pod \"console-d4d69bc5c-nxgx2\" (UID: \"15b0b656-6513-415b-b04a-dddcb2be52ac\") " pod="openshift-console/console-d4d69bc5c-nxgx2" Sep 08 10:09:55.808332 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:55.808257 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15b0b656-6513-415b-b04a-dddcb2be52ac-oauth-serving-cert\") pod \"console-d4d69bc5c-nxgx2\" (UID: \"15b0b656-6513-415b-b04a-dddcb2be52ac\") " pod="openshift-console/console-d4d69bc5c-nxgx2" Sep 08 10:09:55.808332 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:55.808284 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15b0b656-6513-415b-b04a-dddcb2be52ac-service-ca\") pod \"console-d4d69bc5c-nxgx2\" (UID: \"15b0b656-6513-415b-b04a-dddcb2be52ac\") " pod="openshift-console/console-d4d69bc5c-nxgx2" Sep 08 10:09:55.808499 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:55.808336 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qxmd\" (UniqueName: \"kubernetes.io/projected/15b0b656-6513-415b-b04a-dddcb2be52ac-kube-api-access-2qxmd\") pod \"console-d4d69bc5c-nxgx2\" (UID: \"15b0b656-6513-415b-b04a-dddcb2be52ac\") " pod="openshift-console/console-d4d69bc5c-nxgx2" Sep 08 10:09:55.808846 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:55.808813 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15b0b656-6513-415b-b04a-dddcb2be52ac-console-config\") pod \"console-d4d69bc5c-nxgx2\" (UID: \"15b0b656-6513-415b-b04a-dddcb2be52ac\") " pod="openshift-console/console-d4d69bc5c-nxgx2" Sep 08 10:09:55.809293 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:55.809262 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15b0b656-6513-415b-b04a-dddcb2be52ac-oauth-serving-cert\") pod \"console-d4d69bc5c-nxgx2\" (UID: \"15b0b656-6513-415b-b04a-dddcb2be52ac\") " pod="openshift-console/console-d4d69bc5c-nxgx2" Sep 08 10:09:55.809594 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:55.809399 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15b0b656-6513-415b-b04a-dddcb2be52ac-service-ca\") pod \"console-d4d69bc5c-nxgx2\" (UID: \"15b0b656-6513-415b-b04a-dddcb2be52ac\") " pod="openshift-console/console-d4d69bc5c-nxgx2" Sep 08 10:09:55.810049 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:55.810025 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15b0b656-6513-415b-b04a-dddcb2be52ac-trusted-ca-bundle\") pod \"console-d4d69bc5c-nxgx2\" (UID: \"15b0b656-6513-415b-b04a-dddcb2be52ac\") " pod="openshift-console/console-d4d69bc5c-nxgx2" Sep 08 10:09:55.811267 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:55.811244 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15b0b656-6513-415b-b04a-dddcb2be52ac-console-oauth-config\") pod \"console-d4d69bc5c-nxgx2\" (UID: \"15b0b656-6513-415b-b04a-dddcb2be52ac\") " pod="openshift-console/console-d4d69bc5c-nxgx2" Sep 08 10:09:55.811640 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:55.811613 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15b0b656-6513-415b-b04a-dddcb2be52ac-console-serving-cert\") pod \"console-d4d69bc5c-nxgx2\" (UID: \"15b0b656-6513-415b-b04a-dddcb2be52ac\") " pod="openshift-console/console-d4d69bc5c-nxgx2" Sep 08 10:09:55.817286 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:55.817265 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qxmd\" (UniqueName: \"kubernetes.io/projected/15b0b656-6513-415b-b04a-dddcb2be52ac-kube-api-access-2qxmd\") pod \"console-d4d69bc5c-nxgx2\" (UID: \"15b0b656-6513-415b-b04a-dddcb2be52ac\") " pod="openshift-console/console-d4d69bc5c-nxgx2" Sep 08 10:09:55.924523 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:55.924488 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d4d69bc5c-nxgx2" Sep 08 10:09:56.023711 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:56.023676 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zbkhn" event={"ID":"64726051-51dc-4318-a02f-edbc884e3654","Type":"ContainerStarted","Data":"b0757cfc845705885331e4a0d05844126846c0a0a20bffe6caa52080155dd31b"} Sep 08 10:09:56.027212 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:56.027181 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t589v" event={"ID":"042a44d9-65fb-4808-bced-291b70524783","Type":"ContainerStarted","Data":"1d6ef92ebbfeef267f8ed661bd686ad6d418611f848ffa74e3d28f5c209bc2af"} Sep 08 10:09:56.027326 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:56.027231 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t589v" event={"ID":"042a44d9-65fb-4808-bced-291b70524783","Type":"ContainerStarted","Data":"58d60044d6be8cb199e4e230d3be6fdf50990d16fd7eb95d656caf28e9c39820"} Sep 08 10:09:56.043929 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:56.043866 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-zbkhn" podStartSLOduration=2.331948255 podStartE2EDuration="11.043848536s" totalImagesPullingTime="8.711900281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:09:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:09:56.043222062 +0000 UTC m=+180.087326153" watchObservedRunningTime="2026-09-08 10:09:56.043848536 +0000 UTC m=+180.087952618" Sep 08 10:09:56.067331 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:56.067271 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-t589v" podStartSLOduration=2.640399779 podStartE2EDuration="11.067251558s" totalImagesPullingTime="8.353828477s" totalInitContainerRuntime="73.023302ms" isStatefulPod=true podCreationTimestamp="2026-09-08 10:09:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:09:56.065561532 +0000 UTC m=+180.109665625" watchObservedRunningTime="2026-09-08 10:09:56.067251558 +0000 UTC m=+180.111355628" Sep 08 10:09:56.215376 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:56.215344 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d4d69bc5c-nxgx2"] Sep 08 10:09:57.035921 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:57.035875 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d4d69bc5c-nxgx2" event={"ID":"15b0b656-6513-415b-b04a-dddcb2be52ac","Type":"ContainerStarted","Data":"7425edf7d7760ded8d9388a55d09b7dc1583576d0f92854db8bac42a9c90f607"} Sep 08 10:09:57.036356 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:57.035969 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d4d69bc5c-nxgx2" event={"ID":"15b0b656-6513-415b-b04a-dddcb2be52ac","Type":"ContainerStarted","Data":"08ac8ec7ab15216001460fd01e6a5a45b7528befd05750b80054622c58228fca"} Sep 08 10:09:57.055173 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:57.055113 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-d4d69bc5c-nxgx2" podStartSLOduration=2.055090806 podStartE2EDuration="2.055090806s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:09:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:09:57.053047373 +0000 UTC m=+181.097151464" watchObservedRunningTime="2026-09-08 10:09:57.055090806 +0000 UTC m=+181.099194875" Sep 08 10:09:58.041813 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:58.041775 2568 generic.go:356] "Generic (PLEG): container finished" podID="afcc6060-56ad-4294-8d71-c535213cecd2" containerID="8384a466b82782b403423c4470f6a9fa0285dd1cea33ef5b4488166a7fc27f76" exitCode=0 Sep 08 10:09:58.042232 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:09:58.041866 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"afcc6060-56ad-4294-8d71-c535213cecd2","Type":"ContainerDied","Data":"8384a466b82782b403423c4470f6a9fa0285dd1cea33ef5b4488166a7fc27f76"} Sep 08 10:10:00.658562 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:00.657828 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d4d69bc5c-nxgx2"] Sep 08 10:10:00.700894 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:00.700862 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6974cf8d89-m968b"] Sep 08 10:10:00.718872 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:00.718842 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6974cf8d89-m968b" Sep 08 10:10:00.722202 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:00.722180 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6974cf8d89-m968b"] Sep 08 10:10:00.858703 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:00.858680 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e8aa8388-e47a-4160-b614-81d17bede948-console-serving-cert\") pod \"console-6974cf8d89-m968b\" (UID: \"e8aa8388-e47a-4160-b614-81d17bede948\") " pod="openshift-console/console-6974cf8d89-m968b" Sep 08 10:10:00.858819 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:00.858742 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e8aa8388-e47a-4160-b614-81d17bede948-service-ca\") pod \"console-6974cf8d89-m968b\" (UID: \"e8aa8388-e47a-4160-b614-81d17bede948\") " pod="openshift-console/console-6974cf8d89-m968b" Sep 08 10:10:00.858884 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:00.858823 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e8aa8388-e47a-4160-b614-81d17bede948-console-oauth-config\") pod \"console-6974cf8d89-m968b\" (UID: \"e8aa8388-e47a-4160-b614-81d17bede948\") " pod="openshift-console/console-6974cf8d89-m968b" Sep 08 10:10:00.858884 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:00.858852 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e8aa8388-e47a-4160-b614-81d17bede948-oauth-serving-cert\") pod \"console-6974cf8d89-m968b\" (UID: \"e8aa8388-e47a-4160-b614-81d17bede948\") " pod="openshift-console/console-6974cf8d89-m968b" Sep 08 10:10:00.858983 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:00.858911 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e8aa8388-e47a-4160-b614-81d17bede948-console-config\") pod \"console-6974cf8d89-m968b\" (UID: \"e8aa8388-e47a-4160-b614-81d17bede948\") " pod="openshift-console/console-6974cf8d89-m968b" Sep 08 10:10:00.858983 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:00.858944 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e8aa8388-e47a-4160-b614-81d17bede948-trusted-ca-bundle\") pod \"console-6974cf8d89-m968b\" (UID: \"e8aa8388-e47a-4160-b614-81d17bede948\") " pod="openshift-console/console-6974cf8d89-m968b" Sep 08 10:10:00.858983 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:00.858973 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8x8r\" (UniqueName: \"kubernetes.io/projected/e8aa8388-e47a-4160-b614-81d17bede948-kube-api-access-t8x8r\") pod \"console-6974cf8d89-m968b\" (UID: \"e8aa8388-e47a-4160-b614-81d17bede948\") " pod="openshift-console/console-6974cf8d89-m968b" Sep 08 10:10:00.960888 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:00.960314 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e8aa8388-e47a-4160-b614-81d17bede948-console-serving-cert\") pod \"console-6974cf8d89-m968b\" (UID: \"e8aa8388-e47a-4160-b614-81d17bede948\") " pod="openshift-console/console-6974cf8d89-m968b" Sep 08 10:10:00.960888 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:00.960373 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e8aa8388-e47a-4160-b614-81d17bede948-service-ca\") pod \"console-6974cf8d89-m968b\" (UID: \"e8aa8388-e47a-4160-b614-81d17bede948\") " pod="openshift-console/console-6974cf8d89-m968b" Sep 08 10:10:00.960888 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:00.960433 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e8aa8388-e47a-4160-b614-81d17bede948-console-oauth-config\") pod \"console-6974cf8d89-m968b\" (UID: \"e8aa8388-e47a-4160-b614-81d17bede948\") " pod="openshift-console/console-6974cf8d89-m968b" Sep 08 10:10:00.960888 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:00.960457 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e8aa8388-e47a-4160-b614-81d17bede948-oauth-serving-cert\") pod \"console-6974cf8d89-m968b\" (UID: \"e8aa8388-e47a-4160-b614-81d17bede948\") " pod="openshift-console/console-6974cf8d89-m968b" Sep 08 10:10:00.960888 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:00.960506 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e8aa8388-e47a-4160-b614-81d17bede948-console-config\") pod \"console-6974cf8d89-m968b\" (UID: \"e8aa8388-e47a-4160-b614-81d17bede948\") " pod="openshift-console/console-6974cf8d89-m968b" Sep 08 10:10:00.960888 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:00.960539 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e8aa8388-e47a-4160-b614-81d17bede948-trusted-ca-bundle\") pod \"console-6974cf8d89-m968b\" (UID: \"e8aa8388-e47a-4160-b614-81d17bede948\") " pod="openshift-console/console-6974cf8d89-m968b" Sep 08 10:10:00.960888 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:00.960579 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8x8r\" (UniqueName: \"kubernetes.io/projected/e8aa8388-e47a-4160-b614-81d17bede948-kube-api-access-t8x8r\") pod \"console-6974cf8d89-m968b\" (UID: \"e8aa8388-e47a-4160-b614-81d17bede948\") " pod="openshift-console/console-6974cf8d89-m968b" Sep 08 10:10:00.961666 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:00.961638 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e8aa8388-e47a-4160-b614-81d17bede948-oauth-serving-cert\") pod \"console-6974cf8d89-m968b\" (UID: \"e8aa8388-e47a-4160-b614-81d17bede948\") " pod="openshift-console/console-6974cf8d89-m968b" Sep 08 10:10:00.961760 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:00.961739 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e8aa8388-e47a-4160-b614-81d17bede948-console-config\") pod \"console-6974cf8d89-m968b\" (UID: \"e8aa8388-e47a-4160-b614-81d17bede948\") " pod="openshift-console/console-6974cf8d89-m968b" Sep 08 10:10:00.962115 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:00.962094 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e8aa8388-e47a-4160-b614-81d17bede948-service-ca\") pod \"console-6974cf8d89-m968b\" (UID: \"e8aa8388-e47a-4160-b614-81d17bede948\") " pod="openshift-console/console-6974cf8d89-m968b" Sep 08 10:10:00.962572 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:00.962554 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e8aa8388-e47a-4160-b614-81d17bede948-trusted-ca-bundle\") pod \"console-6974cf8d89-m968b\" (UID: \"e8aa8388-e47a-4160-b614-81d17bede948\") " pod="openshift-console/console-6974cf8d89-m968b" Sep 08 10:10:00.963352 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:00.963317 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e8aa8388-e47a-4160-b614-81d17bede948-console-serving-cert\") pod \"console-6974cf8d89-m968b\" (UID: \"e8aa8388-e47a-4160-b614-81d17bede948\") " pod="openshift-console/console-6974cf8d89-m968b" Sep 08 10:10:00.963712 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:00.963691 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e8aa8388-e47a-4160-b614-81d17bede948-console-oauth-config\") pod \"console-6974cf8d89-m968b\" (UID: \"e8aa8388-e47a-4160-b614-81d17bede948\") " pod="openshift-console/console-6974cf8d89-m968b" Sep 08 10:10:00.969851 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:00.969825 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8x8r\" (UniqueName: \"kubernetes.io/projected/e8aa8388-e47a-4160-b614-81d17bede948-kube-api-access-t8x8r\") pod \"console-6974cf8d89-m968b\" (UID: \"e8aa8388-e47a-4160-b614-81d17bede948\") " pod="openshift-console/console-6974cf8d89-m968b" Sep 08 10:10:01.041360 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:01.041326 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6974cf8d89-m968b" Sep 08 10:10:01.060687 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:01.060615 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"afcc6060-56ad-4294-8d71-c535213cecd2","Type":"ContainerStarted","Data":"d13e613b4764ea4ca9402d711a8811633bf525a799a540308b32c5ad51b9417f"} Sep 08 10:10:01.060788 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:01.060696 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"afcc6060-56ad-4294-8d71-c535213cecd2","Type":"ContainerStarted","Data":"724a47ede4a3033b54100ecde9569f03736af93a4929359f6738c3dbbe25ed49"} Sep 08 10:10:01.060788 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:01.060728 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"afcc6060-56ad-4294-8d71-c535213cecd2","Type":"ContainerStarted","Data":"7cbc5d13189b310c6510595aa3290f07b0808287061d5510a759f9f259ec8c33"} Sep 08 10:10:01.060788 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:01.060751 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"afcc6060-56ad-4294-8d71-c535213cecd2","Type":"ContainerStarted","Data":"0d6792f39b2a1af6aa39369346b275fce57f7fb347b16776bb3cf0efec9bc790"} Sep 08 10:10:01.060788 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:01.060775 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"afcc6060-56ad-4294-8d71-c535213cecd2","Type":"ContainerStarted","Data":"2a4e56d2ea4029d58758c2b639e98152ff181a18b2fb01b12a76e58c58763431"} Sep 08 10:10:01.345899 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:01.345870 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6974cf8d89-m968b"] Sep 08 10:10:01.348329 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:10:01.348293 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8aa8388_e47a_4160_b614_81d17bede948.slice/crio-1bb249b27d6af9076b895ad1db9f6eec3ef7b72e6a34a4c0d9dcea567afaa76c WatchSource:0}: Error finding container 1bb249b27d6af9076b895ad1db9f6eec3ef7b72e6a34a4c0d9dcea567afaa76c: Status 404 returned error can't find the container with id 1bb249b27d6af9076b895ad1db9f6eec3ef7b72e6a34a4c0d9dcea567afaa76c Sep 08 10:10:01.588903 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:01.588699 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" podUID="4c12bdd1-d3f2-4659-be40-e653c94d7cbe" containerName="registry" containerID="cri-o://21a6e3f1723433946989592ebd401e8fdd8b8a42ce64a0aa199cf740ea6f2c6c" gracePeriod=30 Sep 08 10:10:01.882975 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:01.882922 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" Sep 08 10:10:01.971523 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:01.971489 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-trusted-ca\") pod \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\" (UID: \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\") " Sep 08 10:10:01.971691 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:01.971546 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-registry-tls\") pod \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\" (UID: \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\") " Sep 08 10:10:01.971691 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:01.971602 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zvnmt\" (UniqueName: \"kubernetes.io/projected/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-kube-api-access-zvnmt\") pod \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\" (UID: \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\") " Sep 08 10:10:01.971691 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:01.971636 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-installation-pull-secrets\") pod \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\" (UID: \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\") " Sep 08 10:10:01.971858 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:01.971704 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-registry-certificates\") pod \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\" (UID: \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\") " Sep 08 10:10:01.971858 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:01.971731 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-ca-trust-extracted\") pod \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\" (UID: \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\") " Sep 08 10:10:01.971858 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:01.971757 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-bound-sa-token\") pod \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\" (UID: \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\") " Sep 08 10:10:01.971858 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:01.971822 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-image-registry-private-configuration\") pod \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\" (UID: \"4c12bdd1-d3f2-4659-be40-e653c94d7cbe\") " Sep 08 10:10:01.972058 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:01.971927 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "4c12bdd1-d3f2-4659-be40-e653c94d7cbe" (UID: "4c12bdd1-d3f2-4659-be40-e653c94d7cbe"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:10:01.972145 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:01.972120 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "4c12bdd1-d3f2-4659-be40-e653c94d7cbe" (UID: "4c12bdd1-d3f2-4659-be40-e653c94d7cbe"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:10:01.972207 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:01.972129 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-trusted-ca\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:10:01.974851 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:01.974819 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "4c12bdd1-d3f2-4659-be40-e653c94d7cbe" (UID: "4c12bdd1-d3f2-4659-be40-e653c94d7cbe"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:10:01.975176 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:01.975135 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "4c12bdd1-d3f2-4659-be40-e653c94d7cbe" (UID: "4c12bdd1-d3f2-4659-be40-e653c94d7cbe"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 10:10:01.975176 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:01.975156 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-kube-api-access-zvnmt" (OuterVolumeSpecName: "kube-api-access-zvnmt") pod "4c12bdd1-d3f2-4659-be40-e653c94d7cbe" (UID: "4c12bdd1-d3f2-4659-be40-e653c94d7cbe"). InnerVolumeSpecName "kube-api-access-zvnmt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 10:10:01.975504 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:01.975479 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "4c12bdd1-d3f2-4659-be40-e653c94d7cbe" (UID: "4c12bdd1-d3f2-4659-be40-e653c94d7cbe"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:10:01.977339 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:01.977315 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4c12bdd1-d3f2-4659-be40-e653c94d7cbe" (UID: "4c12bdd1-d3f2-4659-be40-e653c94d7cbe"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 10:10:01.984737 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:01.984703 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "4c12bdd1-d3f2-4659-be40-e653c94d7cbe" (UID: "4c12bdd1-d3f2-4659-be40-e653c94d7cbe"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 10:10:02.066844 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:02.066810 2568 generic.go:356] "Generic (PLEG): container finished" podID="4c12bdd1-d3f2-4659-be40-e653c94d7cbe" containerID="21a6e3f1723433946989592ebd401e8fdd8b8a42ce64a0aa199cf740ea6f2c6c" exitCode=0 Sep 08 10:10:02.067003 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:02.066859 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" Sep 08 10:10:02.067003 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:02.066897 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" event={"ID":"4c12bdd1-d3f2-4659-be40-e653c94d7cbe","Type":"ContainerDied","Data":"21a6e3f1723433946989592ebd401e8fdd8b8a42ce64a0aa199cf740ea6f2c6c"} Sep 08 10:10:02.067003 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:02.066959 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d54b6c6c4-bxfp6" event={"ID":"4c12bdd1-d3f2-4659-be40-e653c94d7cbe","Type":"ContainerDied","Data":"151840ac476e7082063e922a07f60b50e2e8ab439e144fcd501cc6da3069e1a8"} Sep 08 10:10:02.067003 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:02.066989 2568 scope.go:117] "RemoveContainer" containerID="21a6e3f1723433946989592ebd401e8fdd8b8a42ce64a0aa199cf740ea6f2c6c" Sep 08 10:10:02.069567 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:02.069542 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6974cf8d89-m968b" event={"ID":"e8aa8388-e47a-4160-b614-81d17bede948","Type":"ContainerStarted","Data":"558791f917be10a217d5c72e09b8ecf82875f5d5a2b095b08be9705da2452aa4"} Sep 08 10:10:02.069732 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:02.069707 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6974cf8d89-m968b" event={"ID":"e8aa8388-e47a-4160-b614-81d17bede948","Type":"ContainerStarted","Data":"1bb249b27d6af9076b895ad1db9f6eec3ef7b72e6a34a4c0d9dcea567afaa76c"} Sep 08 10:10:02.074222 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:02.074199 2568 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-registry-certificates\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:10:02.074222 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:02.074224 2568 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-ca-trust-extracted\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:10:02.074364 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:02.074233 2568 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-bound-sa-token\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:10:02.074364 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:02.074243 2568 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-image-registry-private-configuration\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:10:02.074364 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:02.074251 2568 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-registry-tls\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:10:02.074364 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:02.074259 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zvnmt\" (UniqueName: \"kubernetes.io/projected/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-kube-api-access-zvnmt\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:10:02.074364 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:02.074272 2568 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4c12bdd1-d3f2-4659-be40-e653c94d7cbe-installation-pull-secrets\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:10:02.078062 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:02.078040 2568 scope.go:117] "RemoveContainer" containerID="21a6e3f1723433946989592ebd401e8fdd8b8a42ce64a0aa199cf740ea6f2c6c" Sep 08 10:10:02.078352 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:10:02.078330 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21a6e3f1723433946989592ebd401e8fdd8b8a42ce64a0aa199cf740ea6f2c6c\": container with ID starting with 21a6e3f1723433946989592ebd401e8fdd8b8a42ce64a0aa199cf740ea6f2c6c not found: ID does not exist" containerID="21a6e3f1723433946989592ebd401e8fdd8b8a42ce64a0aa199cf740ea6f2c6c" Sep 08 10:10:02.078456 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:02.078359 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21a6e3f1723433946989592ebd401e8fdd8b8a42ce64a0aa199cf740ea6f2c6c"} err="failed to get container status \"21a6e3f1723433946989592ebd401e8fdd8b8a42ce64a0aa199cf740ea6f2c6c\": rpc error: code = NotFound desc = could not find container \"21a6e3f1723433946989592ebd401e8fdd8b8a42ce64a0aa199cf740ea6f2c6c\": container with ID starting with 21a6e3f1723433946989592ebd401e8fdd8b8a42ce64a0aa199cf740ea6f2c6c not found: ID does not exist" Sep 08 10:10:02.094326 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:02.094287 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6974cf8d89-m968b" podStartSLOduration=2.0942762520000002 podStartE2EDuration="2.094276252s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:10:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:10:02.092550302 +0000 UTC m=+186.136654371" watchObservedRunningTime="2026-09-08 10:10:02.094276252 +0000 UTC m=+186.138380319" Sep 08 10:10:02.112973 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:02.112949 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6d54b6c6c4-bxfp6"] Sep 08 10:10:02.117443 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:02.117396 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6d54b6c6c4-bxfp6"] Sep 08 10:10:02.405761 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:02.405732 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c12bdd1-d3f2-4659-be40-e653c94d7cbe" path="/var/lib/kubelet/pods/4c12bdd1-d3f2-4659-be40-e653c94d7cbe/volumes" Sep 08 10:10:04.082244 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:04.082221 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"afcc6060-56ad-4294-8d71-c535213cecd2","Type":"ContainerStarted","Data":"276eb07064786f4387a33aed2ae16939d24a3dfacf87d00e4c3667400d0f8044"} Sep 08 10:10:05.116905 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:05.116856 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=11.185769331 podStartE2EDuration="19.116841282s" totalImagesPullingTime="7.897130129s" totalInitContainerRuntime="33.941822ms" isStatefulPod=false podCreationTimestamp="2026-09-08 10:09:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:10:05.115202987 +0000 UTC m=+189.159307075" watchObservedRunningTime="2026-09-08 10:10:05.116841282 +0000 UTC m=+189.160945349" Sep 08 10:10:05.924885 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:05.924846 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-d4d69bc5c-nxgx2" Sep 08 10:10:11.041910 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:11.041875 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6974cf8d89-m968b" Sep 08 10:10:11.042309 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:11.041943 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6974cf8d89-m968b" Sep 08 10:10:11.046586 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:11.046566 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6974cf8d89-m968b" Sep 08 10:10:11.112187 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:11.112163 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6974cf8d89-m968b" Sep 08 10:10:15.840304 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:15.840262 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-57c9bfb9ff-fpg4s" podUID="2ec9be4b-99b8-495a-a8d8-8c010727c301" containerName="console" containerID="cri-o://67f3cfaf81306b7ec3ff565b71b824ac25ea987f6a0d806fd3f56b602490c59a" gracePeriod=15 Sep 08 10:10:16.111297 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:16.111246 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57c9bfb9ff-fpg4s" Sep 08 10:10:16.129243 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:16.129219 2568 generic.go:356] "Generic (PLEG): container finished" podID="2ec9be4b-99b8-495a-a8d8-8c010727c301" containerID="67f3cfaf81306b7ec3ff565b71b824ac25ea987f6a0d806fd3f56b602490c59a" exitCode=0 Sep 08 10:10:16.129350 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:16.129262 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57c9bfb9ff-fpg4s" Sep 08 10:10:16.129350 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:16.129304 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57c9bfb9ff-fpg4s" event={"ID":"2ec9be4b-99b8-495a-a8d8-8c010727c301","Type":"ContainerDied","Data":"67f3cfaf81306b7ec3ff565b71b824ac25ea987f6a0d806fd3f56b602490c59a"} Sep 08 10:10:16.129459 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:16.129353 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57c9bfb9ff-fpg4s" event={"ID":"2ec9be4b-99b8-495a-a8d8-8c010727c301","Type":"ContainerDied","Data":"3a9f162354cf00621dee66d52e39975aca5581790b1bf4588b0de6c3d7181234"} Sep 08 10:10:16.129459 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:16.129375 2568 scope.go:117] "RemoveContainer" containerID="67f3cfaf81306b7ec3ff565b71b824ac25ea987f6a0d806fd3f56b602490c59a" Sep 08 10:10:16.137311 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:16.137287 2568 scope.go:117] "RemoveContainer" containerID="67f3cfaf81306b7ec3ff565b71b824ac25ea987f6a0d806fd3f56b602490c59a" Sep 08 10:10:16.137879 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:10:16.137858 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67f3cfaf81306b7ec3ff565b71b824ac25ea987f6a0d806fd3f56b602490c59a\": container with ID starting with 67f3cfaf81306b7ec3ff565b71b824ac25ea987f6a0d806fd3f56b602490c59a not found: ID does not exist" containerID="67f3cfaf81306b7ec3ff565b71b824ac25ea987f6a0d806fd3f56b602490c59a" Sep 08 10:10:16.137972 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:16.137889 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67f3cfaf81306b7ec3ff565b71b824ac25ea987f6a0d806fd3f56b602490c59a"} err="failed to get container status \"67f3cfaf81306b7ec3ff565b71b824ac25ea987f6a0d806fd3f56b602490c59a\": rpc error: code = NotFound desc = could not find container \"67f3cfaf81306b7ec3ff565b71b824ac25ea987f6a0d806fd3f56b602490c59a\": container with ID starting with 67f3cfaf81306b7ec3ff565b71b824ac25ea987f6a0d806fd3f56b602490c59a not found: ID does not exist" Sep 08 10:10:16.193529 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:16.193500 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2ec9be4b-99b8-495a-a8d8-8c010727c301-console-oauth-config\") pod \"2ec9be4b-99b8-495a-a8d8-8c010727c301\" (UID: \"2ec9be4b-99b8-495a-a8d8-8c010727c301\") " Sep 08 10:10:16.193690 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:16.193544 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2ec9be4b-99b8-495a-a8d8-8c010727c301-console-serving-cert\") pod \"2ec9be4b-99b8-495a-a8d8-8c010727c301\" (UID: \"2ec9be4b-99b8-495a-a8d8-8c010727c301\") " Sep 08 10:10:16.193690 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:16.193579 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2ec9be4b-99b8-495a-a8d8-8c010727c301-console-config\") pod \"2ec9be4b-99b8-495a-a8d8-8c010727c301\" (UID: \"2ec9be4b-99b8-495a-a8d8-8c010727c301\") " Sep 08 10:10:16.193690 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:16.193639 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2ec9be4b-99b8-495a-a8d8-8c010727c301-service-ca\") pod \"2ec9be4b-99b8-495a-a8d8-8c010727c301\" (UID: \"2ec9be4b-99b8-495a-a8d8-8c010727c301\") " Sep 08 10:10:16.193690 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:16.193660 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2ec9be4b-99b8-495a-a8d8-8c010727c301-oauth-serving-cert\") pod \"2ec9be4b-99b8-495a-a8d8-8c010727c301\" (UID: \"2ec9be4b-99b8-495a-a8d8-8c010727c301\") " Sep 08 10:10:16.193877 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:16.193694 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4vpc7\" (UniqueName: \"kubernetes.io/projected/2ec9be4b-99b8-495a-a8d8-8c010727c301-kube-api-access-4vpc7\") pod \"2ec9be4b-99b8-495a-a8d8-8c010727c301\" (UID: \"2ec9be4b-99b8-495a-a8d8-8c010727c301\") " Sep 08 10:10:16.194047 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:16.194020 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ec9be4b-99b8-495a-a8d8-8c010727c301-console-config" (OuterVolumeSpecName: "console-config") pod "2ec9be4b-99b8-495a-a8d8-8c010727c301" (UID: "2ec9be4b-99b8-495a-a8d8-8c010727c301"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:10:16.194117 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:16.194044 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ec9be4b-99b8-495a-a8d8-8c010727c301-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2ec9be4b-99b8-495a-a8d8-8c010727c301" (UID: "2ec9be4b-99b8-495a-a8d8-8c010727c301"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:10:16.194117 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:16.194057 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ec9be4b-99b8-495a-a8d8-8c010727c301-service-ca" (OuterVolumeSpecName: "service-ca") pod "2ec9be4b-99b8-495a-a8d8-8c010727c301" (UID: "2ec9be4b-99b8-495a-a8d8-8c010727c301"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:10:16.196133 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:16.196104 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ec9be4b-99b8-495a-a8d8-8c010727c301-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2ec9be4b-99b8-495a-a8d8-8c010727c301" (UID: "2ec9be4b-99b8-495a-a8d8-8c010727c301"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:10:16.196250 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:16.196218 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ec9be4b-99b8-495a-a8d8-8c010727c301-kube-api-access-4vpc7" (OuterVolumeSpecName: "kube-api-access-4vpc7") pod "2ec9be4b-99b8-495a-a8d8-8c010727c301" (UID: "2ec9be4b-99b8-495a-a8d8-8c010727c301"). InnerVolumeSpecName "kube-api-access-4vpc7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 10:10:16.196369 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:16.196351 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ec9be4b-99b8-495a-a8d8-8c010727c301-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2ec9be4b-99b8-495a-a8d8-8c010727c301" (UID: "2ec9be4b-99b8-495a-a8d8-8c010727c301"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:10:16.294975 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:16.294948 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2ec9be4b-99b8-495a-a8d8-8c010727c301-console-oauth-config\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:10:16.294975 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:16.294968 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2ec9be4b-99b8-495a-a8d8-8c010727c301-console-serving-cert\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:10:16.294975 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:16.294977 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2ec9be4b-99b8-495a-a8d8-8c010727c301-console-config\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:10:16.295156 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:16.294985 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2ec9be4b-99b8-495a-a8d8-8c010727c301-service-ca\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:10:16.295156 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:16.294993 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2ec9be4b-99b8-495a-a8d8-8c010727c301-oauth-serving-cert\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:10:16.295156 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:16.295001 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4vpc7\" (UniqueName: \"kubernetes.io/projected/2ec9be4b-99b8-495a-a8d8-8c010727c301-kube-api-access-4vpc7\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:10:16.446926 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:16.446905 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57c9bfb9ff-fpg4s"] Sep 08 10:10:16.449103 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:16.449080 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-57c9bfb9ff-fpg4s"] Sep 08 10:10:18.139336 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:18.139312 2568 generic.go:356] "Generic (PLEG): container finished" podID="1373d83d-e33c-4993-bfe5-7ff0b9241a79" containerID="3bdf0f114aac8e1a7b586e499185b93839ad4d0da9eb0af533310d974e0cf0b8" exitCode=0 Sep 08 10:10:18.139639 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:18.139347 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bjv7p" event={"ID":"1373d83d-e33c-4993-bfe5-7ff0b9241a79","Type":"ContainerDied","Data":"3bdf0f114aac8e1a7b586e499185b93839ad4d0da9eb0af533310d974e0cf0b8"} Sep 08 10:10:18.139639 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:18.139633 2568 scope.go:117] "RemoveContainer" containerID="3bdf0f114aac8e1a7b586e499185b93839ad4d0da9eb0af533310d974e0cf0b8" Sep 08 10:10:18.404765 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:18.404735 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ec9be4b-99b8-495a-a8d8-8c010727c301" path="/var/lib/kubelet/pods/2ec9be4b-99b8-495a-a8d8-8c010727c301/volumes" Sep 08 10:10:19.145617 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:19.145589 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bjv7p" event={"ID":"1373d83d-e33c-4993-bfe5-7ff0b9241a79","Type":"ContainerStarted","Data":"e057c7fbdffb9751a94aebcb1518fdbd209c3676fa10b23976b4c1d88c57294d"} Sep 08 10:10:25.678869 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:25.678826 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-d4d69bc5c-nxgx2" podUID="15b0b656-6513-415b-b04a-dddcb2be52ac" containerName="console" containerID="cri-o://7425edf7d7760ded8d9388a55d09b7dc1583576d0f92854db8bac42a9c90f607" gracePeriod=15 Sep 08 10:10:25.938860 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:25.938831 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d4d69bc5c-nxgx2" Sep 08 10:10:25.959338 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:25.959313 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15b0b656-6513-415b-b04a-dddcb2be52ac-console-oauth-config\") pod \"15b0b656-6513-415b-b04a-dddcb2be52ac\" (UID: \"15b0b656-6513-415b-b04a-dddcb2be52ac\") " Sep 08 10:10:25.959473 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:25.959346 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15b0b656-6513-415b-b04a-dddcb2be52ac-oauth-serving-cert\") pod \"15b0b656-6513-415b-b04a-dddcb2be52ac\" (UID: \"15b0b656-6513-415b-b04a-dddcb2be52ac\") " Sep 08 10:10:25.959473 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:25.959370 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2qxmd\" (UniqueName: \"kubernetes.io/projected/15b0b656-6513-415b-b04a-dddcb2be52ac-kube-api-access-2qxmd\") pod \"15b0b656-6513-415b-b04a-dddcb2be52ac\" (UID: \"15b0b656-6513-415b-b04a-dddcb2be52ac\") " Sep 08 10:10:25.959473 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:25.959394 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15b0b656-6513-415b-b04a-dddcb2be52ac-console-config\") pod \"15b0b656-6513-415b-b04a-dddcb2be52ac\" (UID: \"15b0b656-6513-415b-b04a-dddcb2be52ac\") " Sep 08 10:10:25.959473 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:25.959445 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15b0b656-6513-415b-b04a-dddcb2be52ac-console-serving-cert\") pod \"15b0b656-6513-415b-b04a-dddcb2be52ac\" (UID: \"15b0b656-6513-415b-b04a-dddcb2be52ac\") " Sep 08 10:10:25.959668 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:25.959483 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15b0b656-6513-415b-b04a-dddcb2be52ac-service-ca\") pod \"15b0b656-6513-415b-b04a-dddcb2be52ac\" (UID: \"15b0b656-6513-415b-b04a-dddcb2be52ac\") " Sep 08 10:10:25.959668 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:25.959524 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15b0b656-6513-415b-b04a-dddcb2be52ac-trusted-ca-bundle\") pod \"15b0b656-6513-415b-b04a-dddcb2be52ac\" (UID: \"15b0b656-6513-415b-b04a-dddcb2be52ac\") " Sep 08 10:10:25.959890 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:25.959855 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15b0b656-6513-415b-b04a-dddcb2be52ac-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "15b0b656-6513-415b-b04a-dddcb2be52ac" (UID: "15b0b656-6513-415b-b04a-dddcb2be52ac"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:10:25.960002 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:25.959976 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15b0b656-6513-415b-b04a-dddcb2be52ac-console-config" (OuterVolumeSpecName: "console-config") pod "15b0b656-6513-415b-b04a-dddcb2be52ac" (UID: "15b0b656-6513-415b-b04a-dddcb2be52ac"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:10:25.960144 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:25.960119 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15b0b656-6513-415b-b04a-dddcb2be52ac-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "15b0b656-6513-415b-b04a-dddcb2be52ac" (UID: "15b0b656-6513-415b-b04a-dddcb2be52ac"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:10:25.960206 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:25.960122 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15b0b656-6513-415b-b04a-dddcb2be52ac-service-ca" (OuterVolumeSpecName: "service-ca") pod "15b0b656-6513-415b-b04a-dddcb2be52ac" (UID: "15b0b656-6513-415b-b04a-dddcb2be52ac"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:10:25.961609 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:25.961588 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15b0b656-6513-415b-b04a-dddcb2be52ac-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "15b0b656-6513-415b-b04a-dddcb2be52ac" (UID: "15b0b656-6513-415b-b04a-dddcb2be52ac"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:10:25.961783 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:25.961764 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15b0b656-6513-415b-b04a-dddcb2be52ac-kube-api-access-2qxmd" (OuterVolumeSpecName: "kube-api-access-2qxmd") pod "15b0b656-6513-415b-b04a-dddcb2be52ac" (UID: "15b0b656-6513-415b-b04a-dddcb2be52ac"). InnerVolumeSpecName "kube-api-access-2qxmd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 10:10:25.961882 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:25.961865 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15b0b656-6513-415b-b04a-dddcb2be52ac-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "15b0b656-6513-415b-b04a-dddcb2be52ac" (UID: "15b0b656-6513-415b-b04a-dddcb2be52ac"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:10:26.060272 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:26.060215 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15b0b656-6513-415b-b04a-dddcb2be52ac-oauth-serving-cert\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:10:26.060272 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:26.060237 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qxmd\" (UniqueName: \"kubernetes.io/projected/15b0b656-6513-415b-b04a-dddcb2be52ac-kube-api-access-2qxmd\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:10:26.060272 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:26.060247 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15b0b656-6513-415b-b04a-dddcb2be52ac-console-config\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:10:26.060272 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:26.060255 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15b0b656-6513-415b-b04a-dddcb2be52ac-console-serving-cert\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:10:26.060272 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:26.060263 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15b0b656-6513-415b-b04a-dddcb2be52ac-service-ca\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:10:26.060272 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:26.060272 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15b0b656-6513-415b-b04a-dddcb2be52ac-trusted-ca-bundle\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:10:26.060510 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:26.060279 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15b0b656-6513-415b-b04a-dddcb2be52ac-console-oauth-config\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:10:26.171873 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:26.171839 2568 generic.go:356] "Generic (PLEG): container finished" podID="15b0b656-6513-415b-b04a-dddcb2be52ac" containerID="7425edf7d7760ded8d9388a55d09b7dc1583576d0f92854db8bac42a9c90f607" exitCode=0 Sep 08 10:10:26.171970 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:26.171888 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d4d69bc5c-nxgx2" Sep 08 10:10:26.171970 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:26.171926 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d4d69bc5c-nxgx2" event={"ID":"15b0b656-6513-415b-b04a-dddcb2be52ac","Type":"ContainerDied","Data":"7425edf7d7760ded8d9388a55d09b7dc1583576d0f92854db8bac42a9c90f607"} Sep 08 10:10:26.172059 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:26.171983 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d4d69bc5c-nxgx2" event={"ID":"15b0b656-6513-415b-b04a-dddcb2be52ac","Type":"ContainerDied","Data":"08ac8ec7ab15216001460fd01e6a5a45b7528befd05750b80054622c58228fca"} Sep 08 10:10:26.172059 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:26.172013 2568 scope.go:117] "RemoveContainer" containerID="7425edf7d7760ded8d9388a55d09b7dc1583576d0f92854db8bac42a9c90f607" Sep 08 10:10:26.173570 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:26.173550 2568 generic.go:356] "Generic (PLEG): container finished" podID="852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a" containerID="19b2f751294924eedd1ef3cb52043dd6975f198583897fd9ebd8be459272e180" exitCode=0 Sep 08 10:10:26.173753 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:26.173703 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-524nf" event={"ID":"852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a","Type":"ContainerDied","Data":"19b2f751294924eedd1ef3cb52043dd6975f198583897fd9ebd8be459272e180"} Sep 08 10:10:26.174107 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:26.174091 2568 scope.go:117] "RemoveContainer" containerID="19b2f751294924eedd1ef3cb52043dd6975f198583897fd9ebd8be459272e180" Sep 08 10:10:26.180649 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:26.180630 2568 scope.go:117] "RemoveContainer" containerID="7425edf7d7760ded8d9388a55d09b7dc1583576d0f92854db8bac42a9c90f607" Sep 08 10:10:26.180947 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:10:26.180923 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7425edf7d7760ded8d9388a55d09b7dc1583576d0f92854db8bac42a9c90f607\": container with ID starting with 7425edf7d7760ded8d9388a55d09b7dc1583576d0f92854db8bac42a9c90f607 not found: ID does not exist" containerID="7425edf7d7760ded8d9388a55d09b7dc1583576d0f92854db8bac42a9c90f607" Sep 08 10:10:26.181026 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:26.180950 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7425edf7d7760ded8d9388a55d09b7dc1583576d0f92854db8bac42a9c90f607"} err="failed to get container status \"7425edf7d7760ded8d9388a55d09b7dc1583576d0f92854db8bac42a9c90f607\": rpc error: code = NotFound desc = could not find container \"7425edf7d7760ded8d9388a55d09b7dc1583576d0f92854db8bac42a9c90f607\": container with ID starting with 7425edf7d7760ded8d9388a55d09b7dc1583576d0f92854db8bac42a9c90f607 not found: ID does not exist" Sep 08 10:10:26.235021 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:26.234993 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d4d69bc5c-nxgx2"] Sep 08 10:10:26.240476 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:26.240451 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-d4d69bc5c-nxgx2"] Sep 08 10:10:26.409798 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:26.409755 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15b0b656-6513-415b-b04a-dddcb2be52ac" path="/var/lib/kubelet/pods/15b0b656-6513-415b-b04a-dddcb2be52ac/volumes" Sep 08 10:10:27.180273 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:10:27.180236 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-524nf" event={"ID":"852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a","Type":"ContainerStarted","Data":"2b5673bb5c34c5d5a76a7518f3c111536416b1a92ff37e8b8671ef1c14d55c11"} Sep 08 10:11:07.170925 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:07.170892 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b4e4397-1898-4201-9dcd-b5775ee1789c-metrics-certs\") pod \"network-metrics-daemon-mbs64\" (UID: \"3b4e4397-1898-4201-9dcd-b5775ee1789c\") " pod="openshift-multus/network-metrics-daemon-mbs64" Sep 08 10:11:07.173232 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:07.173210 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b4e4397-1898-4201-9dcd-b5775ee1789c-metrics-certs\") pod \"network-metrics-daemon-mbs64\" (UID: \"3b4e4397-1898-4201-9dcd-b5775ee1789c\") " pod="openshift-multus/network-metrics-daemon-mbs64" Sep 08 10:11:07.302228 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:07.302205 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tm4jr\"" Sep 08 10:11:07.309475 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:07.309455 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbs64" Sep 08 10:11:07.599681 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:07.599614 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mbs64"] Sep 08 10:11:07.606128 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:11:07.606101 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b4e4397_1898_4201_9dcd_b5775ee1789c.slice/crio-9f5f5eac4be2a732bd0d9ec7d922b5d5b31aafb7eacb1de80c6ff1ad6380507a WatchSource:0}: Error finding container 9f5f5eac4be2a732bd0d9ec7d922b5d5b31aafb7eacb1de80c6ff1ad6380507a: Status 404 returned error can't find the container with id 9f5f5eac4be2a732bd0d9ec7d922b5d5b31aafb7eacb1de80c6ff1ad6380507a Sep 08 10:11:08.343302 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:08.343264 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mbs64" event={"ID":"3b4e4397-1898-4201-9dcd-b5775ee1789c","Type":"ContainerStarted","Data":"9f5f5eac4be2a732bd0d9ec7d922b5d5b31aafb7eacb1de80c6ff1ad6380507a"} Sep 08 10:11:09.348732 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:09.348692 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mbs64" event={"ID":"3b4e4397-1898-4201-9dcd-b5775ee1789c","Type":"ContainerStarted","Data":"a9bdb750f0a4d4835e1952ec685dfdd129846f784605bfaec2b10b851c1314de"} Sep 08 10:11:09.349077 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:09.348753 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mbs64" event={"ID":"3b4e4397-1898-4201-9dcd-b5775ee1789c","Type":"ContainerStarted","Data":"925fa235f4038df92afa12878f56d4e6ec1a89f18ab1e5a212df53e9a7af193d"} Sep 08 10:11:09.381431 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:09.381363 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-mbs64" podStartSLOduration=252.452805822 podStartE2EDuration="4m13.381349658s" totalImagesPullingTime="928.543836ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:06:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:11:09.380687334 +0000 UTC m=+253.424791422" watchObservedRunningTime="2026-09-08 10:11:09.381349658 +0000 UTC m=+253.425453726" Sep 08 10:11:11.407648 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:11.407621 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d6848b855-nkqs6"] Sep 08 10:11:11.408036 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:11.407950 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ec9be4b-99b8-495a-a8d8-8c010727c301" containerName="console" Sep 08 10:11:11.408036 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:11.407964 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ec9be4b-99b8-495a-a8d8-8c010727c301" containerName="console" Sep 08 10:11:11.408036 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:11.407975 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15b0b656-6513-415b-b04a-dddcb2be52ac" containerName="console" Sep 08 10:11:11.408036 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:11.407980 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="15b0b656-6513-415b-b04a-dddcb2be52ac" containerName="console" Sep 08 10:11:11.408036 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:11.407995 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c12bdd1-d3f2-4659-be40-e653c94d7cbe" containerName="registry" Sep 08 10:11:11.408036 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:11.408001 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c12bdd1-d3f2-4659-be40-e653c94d7cbe" containerName="registry" Sep 08 10:11:11.408218 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:11.408046 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c12bdd1-d3f2-4659-be40-e653c94d7cbe" containerName="registry" Sep 08 10:11:11.408218 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:11.408055 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ec9be4b-99b8-495a-a8d8-8c010727c301" containerName="console" Sep 08 10:11:11.408218 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:11.408062 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="15b0b656-6513-415b-b04a-dddcb2be52ac" containerName="console" Sep 08 10:11:11.409821 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:11.409805 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d6848b855-nkqs6" Sep 08 10:11:11.450945 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:11.450916 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d6848b855-nkqs6"] Sep 08 10:11:11.505465 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:11.505403 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-console-oauth-config\") pod \"console-7d6848b855-nkqs6\" (UID: \"beda3813-0e7a-4cbc-9d03-2dcc71cb867c\") " pod="openshift-console/console-7d6848b855-nkqs6" Sep 08 10:11:11.505465 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:11.505466 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvwmn\" (UniqueName: \"kubernetes.io/projected/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-kube-api-access-nvwmn\") pod \"console-7d6848b855-nkqs6\" (UID: \"beda3813-0e7a-4cbc-9d03-2dcc71cb867c\") " pod="openshift-console/console-7d6848b855-nkqs6" Sep 08 10:11:11.505666 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:11.505497 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-console-config\") pod \"console-7d6848b855-nkqs6\" (UID: \"beda3813-0e7a-4cbc-9d03-2dcc71cb867c\") " pod="openshift-console/console-7d6848b855-nkqs6" Sep 08 10:11:11.505666 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:11.505537 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-console-serving-cert\") pod \"console-7d6848b855-nkqs6\" (UID: \"beda3813-0e7a-4cbc-9d03-2dcc71cb867c\") " pod="openshift-console/console-7d6848b855-nkqs6" Sep 08 10:11:11.505666 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:11.505560 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-trusted-ca-bundle\") pod \"console-7d6848b855-nkqs6\" (UID: \"beda3813-0e7a-4cbc-9d03-2dcc71cb867c\") " pod="openshift-console/console-7d6848b855-nkqs6" Sep 08 10:11:11.505666 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:11.505614 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-oauth-serving-cert\") pod \"console-7d6848b855-nkqs6\" (UID: \"beda3813-0e7a-4cbc-9d03-2dcc71cb867c\") " pod="openshift-console/console-7d6848b855-nkqs6" Sep 08 10:11:11.505666 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:11.505657 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-service-ca\") pod \"console-7d6848b855-nkqs6\" (UID: \"beda3813-0e7a-4cbc-9d03-2dcc71cb867c\") " pod="openshift-console/console-7d6848b855-nkqs6" Sep 08 10:11:11.607022 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:11.606989 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-console-oauth-config\") pod \"console-7d6848b855-nkqs6\" (UID: \"beda3813-0e7a-4cbc-9d03-2dcc71cb867c\") " pod="openshift-console/console-7d6848b855-nkqs6" Sep 08 10:11:11.607108 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:11.607028 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvwmn\" (UniqueName: \"kubernetes.io/projected/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-kube-api-access-nvwmn\") pod \"console-7d6848b855-nkqs6\" (UID: \"beda3813-0e7a-4cbc-9d03-2dcc71cb867c\") " pod="openshift-console/console-7d6848b855-nkqs6" Sep 08 10:11:11.607108 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:11.607047 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-console-config\") pod \"console-7d6848b855-nkqs6\" (UID: \"beda3813-0e7a-4cbc-9d03-2dcc71cb867c\") " pod="openshift-console/console-7d6848b855-nkqs6" Sep 08 10:11:11.607108 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:11.607076 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-console-serving-cert\") pod \"console-7d6848b855-nkqs6\" (UID: \"beda3813-0e7a-4cbc-9d03-2dcc71cb867c\") " pod="openshift-console/console-7d6848b855-nkqs6" Sep 08 10:11:11.607108 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:11.607102 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-trusted-ca-bundle\") pod \"console-7d6848b855-nkqs6\" (UID: \"beda3813-0e7a-4cbc-9d03-2dcc71cb867c\") " pod="openshift-console/console-7d6848b855-nkqs6" Sep 08 10:11:11.607262 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:11.607134 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-oauth-serving-cert\") pod \"console-7d6848b855-nkqs6\" (UID: \"beda3813-0e7a-4cbc-9d03-2dcc71cb867c\") " pod="openshift-console/console-7d6848b855-nkqs6" Sep 08 10:11:11.607262 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:11.607175 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-service-ca\") pod \"console-7d6848b855-nkqs6\" (UID: \"beda3813-0e7a-4cbc-9d03-2dcc71cb867c\") " pod="openshift-console/console-7d6848b855-nkqs6" Sep 08 10:11:11.607895 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:11.607856 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-console-config\") pod \"console-7d6848b855-nkqs6\" (UID: \"beda3813-0e7a-4cbc-9d03-2dcc71cb867c\") " pod="openshift-console/console-7d6848b855-nkqs6" Sep 08 10:11:11.607895 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:11.607866 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-oauth-serving-cert\") pod \"console-7d6848b855-nkqs6\" (UID: \"beda3813-0e7a-4cbc-9d03-2dcc71cb867c\") " pod="openshift-console/console-7d6848b855-nkqs6" Sep 08 10:11:11.608066 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:11.607902 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-service-ca\") pod \"console-7d6848b855-nkqs6\" (UID: \"beda3813-0e7a-4cbc-9d03-2dcc71cb867c\") " pod="openshift-console/console-7d6848b855-nkqs6" Sep 08 10:11:11.608066 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:11.608000 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-trusted-ca-bundle\") pod \"console-7d6848b855-nkqs6\" (UID: \"beda3813-0e7a-4cbc-9d03-2dcc71cb867c\") " pod="openshift-console/console-7d6848b855-nkqs6" Sep 08 10:11:11.609732 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:11.609701 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-console-oauth-config\") pod \"console-7d6848b855-nkqs6\" (UID: \"beda3813-0e7a-4cbc-9d03-2dcc71cb867c\") " pod="openshift-console/console-7d6848b855-nkqs6" Sep 08 10:11:11.609830 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:11.609750 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-console-serving-cert\") pod \"console-7d6848b855-nkqs6\" (UID: \"beda3813-0e7a-4cbc-9d03-2dcc71cb867c\") " pod="openshift-console/console-7d6848b855-nkqs6" Sep 08 10:11:11.623008 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:11.622984 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvwmn\" (UniqueName: \"kubernetes.io/projected/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-kube-api-access-nvwmn\") pod \"console-7d6848b855-nkqs6\" (UID: \"beda3813-0e7a-4cbc-9d03-2dcc71cb867c\") " pod="openshift-console/console-7d6848b855-nkqs6" Sep 08 10:11:11.718285 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:11.718196 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d6848b855-nkqs6" Sep 08 10:11:12.017731 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:12.017634 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d6848b855-nkqs6"] Sep 08 10:11:12.020644 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:11:12.020613 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbeda3813_0e7a_4cbc_9d03_2dcc71cb867c.slice/crio-c360128cdd4047f95dea9d7ccee16a604a3881f140c545729c297df873e4ea5a WatchSource:0}: Error finding container c360128cdd4047f95dea9d7ccee16a604a3881f140c545729c297df873e4ea5a: Status 404 returned error can't find the container with id c360128cdd4047f95dea9d7ccee16a604a3881f140c545729c297df873e4ea5a Sep 08 10:11:12.366364 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:12.366265 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d6848b855-nkqs6" event={"ID":"beda3813-0e7a-4cbc-9d03-2dcc71cb867c","Type":"ContainerStarted","Data":"15343b4770e3c896876d92ea61eb0f7154a01cb9bc2755f919020af9daa6aab6"} Sep 08 10:11:12.366552 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:12.366367 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d6848b855-nkqs6" event={"ID":"beda3813-0e7a-4cbc-9d03-2dcc71cb867c","Type":"ContainerStarted","Data":"c360128cdd4047f95dea9d7ccee16a604a3881f140c545729c297df873e4ea5a"} Sep 08 10:11:21.719362 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:21.719285 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d6848b855-nkqs6" Sep 08 10:11:21.719745 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:21.719557 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7d6848b855-nkqs6" Sep 08 10:11:21.724012 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:21.723989 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7d6848b855-nkqs6" Sep 08 10:11:21.751343 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:21.751299 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7d6848b855-nkqs6" podStartSLOduration=10.751284075000001 podStartE2EDuration="10.751284075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:11:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:11:12.411522222 +0000 UTC m=+256.455626310" watchObservedRunningTime="2026-09-08 10:11:21.751284075 +0000 UTC m=+265.795388143" Sep 08 10:11:22.406193 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:22.406169 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7d6848b855-nkqs6" Sep 08 10:11:22.516744 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:22.516716 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6974cf8d89-m968b"] Sep 08 10:11:47.540575 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:47.540528 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6974cf8d89-m968b" podUID="e8aa8388-e47a-4160-b614-81d17bede948" containerName="console" containerID="cri-o://558791f917be10a217d5c72e09b8ecf82875f5d5a2b095b08be9705da2452aa4" gracePeriod=15 Sep 08 10:11:47.774651 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:47.774629 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6974cf8d89-m968b" Sep 08 10:11:47.959797 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:47.959779 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e8aa8388-e47a-4160-b614-81d17bede948-service-ca\") pod \"e8aa8388-e47a-4160-b614-81d17bede948\" (UID: \"e8aa8388-e47a-4160-b614-81d17bede948\") " Sep 08 10:11:47.959927 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:47.959814 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e8aa8388-e47a-4160-b614-81d17bede948-trusted-ca-bundle\") pod \"e8aa8388-e47a-4160-b614-81d17bede948\" (UID: \"e8aa8388-e47a-4160-b614-81d17bede948\") " Sep 08 10:11:47.959927 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:47.959840 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e8aa8388-e47a-4160-b614-81d17bede948-console-serving-cert\") pod \"e8aa8388-e47a-4160-b614-81d17bede948\" (UID: \"e8aa8388-e47a-4160-b614-81d17bede948\") " Sep 08 10:11:47.960041 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:47.960017 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e8aa8388-e47a-4160-b614-81d17bede948-console-oauth-config\") pod \"e8aa8388-e47a-4160-b614-81d17bede948\" (UID: \"e8aa8388-e47a-4160-b614-81d17bede948\") " Sep 08 10:11:47.960081 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:47.960071 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e8aa8388-e47a-4160-b614-81d17bede948-oauth-serving-cert\") pod \"e8aa8388-e47a-4160-b614-81d17bede948\" (UID: \"e8aa8388-e47a-4160-b614-81d17bede948\") " Sep 08 10:11:47.960128 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:47.960110 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t8x8r\" (UniqueName: \"kubernetes.io/projected/e8aa8388-e47a-4160-b614-81d17bede948-kube-api-access-t8x8r\") pod \"e8aa8388-e47a-4160-b614-81d17bede948\" (UID: \"e8aa8388-e47a-4160-b614-81d17bede948\") " Sep 08 10:11:47.960179 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:47.960158 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e8aa8388-e47a-4160-b614-81d17bede948-console-config\") pod \"e8aa8388-e47a-4160-b614-81d17bede948\" (UID: \"e8aa8388-e47a-4160-b614-81d17bede948\") " Sep 08 10:11:47.960352 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:47.960154 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8aa8388-e47a-4160-b614-81d17bede948-service-ca" (OuterVolumeSpecName: "service-ca") pod "e8aa8388-e47a-4160-b614-81d17bede948" (UID: "e8aa8388-e47a-4160-b614-81d17bede948"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:11:47.960352 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:47.960220 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8aa8388-e47a-4160-b614-81d17bede948-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e8aa8388-e47a-4160-b614-81d17bede948" (UID: "e8aa8388-e47a-4160-b614-81d17bede948"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:11:47.960523 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:47.960388 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e8aa8388-e47a-4160-b614-81d17bede948-service-ca\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:11:47.960523 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:47.960405 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e8aa8388-e47a-4160-b614-81d17bede948-trusted-ca-bundle\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:11:47.960523 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:47.960474 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8aa8388-e47a-4160-b614-81d17bede948-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e8aa8388-e47a-4160-b614-81d17bede948" (UID: "e8aa8388-e47a-4160-b614-81d17bede948"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:11:47.960632 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:47.960533 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8aa8388-e47a-4160-b614-81d17bede948-console-config" (OuterVolumeSpecName: "console-config") pod "e8aa8388-e47a-4160-b614-81d17bede948" (UID: "e8aa8388-e47a-4160-b614-81d17bede948"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:11:47.962057 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:47.962036 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8aa8388-e47a-4160-b614-81d17bede948-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e8aa8388-e47a-4160-b614-81d17bede948" (UID: "e8aa8388-e47a-4160-b614-81d17bede948"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:11:47.962057 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:47.962050 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8aa8388-e47a-4160-b614-81d17bede948-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e8aa8388-e47a-4160-b614-81d17bede948" (UID: "e8aa8388-e47a-4160-b614-81d17bede948"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:11:47.962454 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:47.962437 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8aa8388-e47a-4160-b614-81d17bede948-kube-api-access-t8x8r" (OuterVolumeSpecName: "kube-api-access-t8x8r") pod "e8aa8388-e47a-4160-b614-81d17bede948" (UID: "e8aa8388-e47a-4160-b614-81d17bede948"). InnerVolumeSpecName "kube-api-access-t8x8r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 10:11:48.061178 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:48.061150 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8x8r\" (UniqueName: \"kubernetes.io/projected/e8aa8388-e47a-4160-b614-81d17bede948-kube-api-access-t8x8r\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:11:48.061178 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:48.061177 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e8aa8388-e47a-4160-b614-81d17bede948-console-config\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:11:48.061178 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:48.061188 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e8aa8388-e47a-4160-b614-81d17bede948-console-serving-cert\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:11:48.061333 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:48.061196 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e8aa8388-e47a-4160-b614-81d17bede948-console-oauth-config\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:11:48.061333 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:48.061205 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e8aa8388-e47a-4160-b614-81d17bede948-oauth-serving-cert\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:11:48.506391 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:48.506362 2568 generic.go:356] "Generic (PLEG): container finished" podID="e8aa8388-e47a-4160-b614-81d17bede948" containerID="558791f917be10a217d5c72e09b8ecf82875f5d5a2b095b08be9705da2452aa4" exitCode=0 Sep 08 10:11:48.506553 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:48.506434 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6974cf8d89-m968b" Sep 08 10:11:48.506553 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:48.506446 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6974cf8d89-m968b" event={"ID":"e8aa8388-e47a-4160-b614-81d17bede948","Type":"ContainerDied","Data":"558791f917be10a217d5c72e09b8ecf82875f5d5a2b095b08be9705da2452aa4"} Sep 08 10:11:48.506553 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:48.506542 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6974cf8d89-m968b" event={"ID":"e8aa8388-e47a-4160-b614-81d17bede948","Type":"ContainerDied","Data":"1bb249b27d6af9076b895ad1db9f6eec3ef7b72e6a34a4c0d9dcea567afaa76c"} Sep 08 10:11:48.506714 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:48.506571 2568 scope.go:117] "RemoveContainer" containerID="558791f917be10a217d5c72e09b8ecf82875f5d5a2b095b08be9705da2452aa4" Sep 08 10:11:48.514705 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:48.514688 2568 scope.go:117] "RemoveContainer" containerID="558791f917be10a217d5c72e09b8ecf82875f5d5a2b095b08be9705da2452aa4" Sep 08 10:11:48.514980 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:11:48.514959 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"558791f917be10a217d5c72e09b8ecf82875f5d5a2b095b08be9705da2452aa4\": container with ID starting with 558791f917be10a217d5c72e09b8ecf82875f5d5a2b095b08be9705da2452aa4 not found: ID does not exist" containerID="558791f917be10a217d5c72e09b8ecf82875f5d5a2b095b08be9705da2452aa4" Sep 08 10:11:48.515042 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:48.514997 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"558791f917be10a217d5c72e09b8ecf82875f5d5a2b095b08be9705da2452aa4"} err="failed to get container status \"558791f917be10a217d5c72e09b8ecf82875f5d5a2b095b08be9705da2452aa4\": rpc error: code = NotFound desc = could not find container \"558791f917be10a217d5c72e09b8ecf82875f5d5a2b095b08be9705da2452aa4\": container with ID starting with 558791f917be10a217d5c72e09b8ecf82875f5d5a2b095b08be9705da2452aa4 not found: ID does not exist" Sep 08 10:11:48.524330 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:48.524308 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6974cf8d89-m968b"] Sep 08 10:11:48.530011 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:48.529994 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6974cf8d89-m968b"] Sep 08 10:11:50.403266 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:50.403233 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8aa8388-e47a-4160-b614-81d17bede948" path="/var/lib/kubelet/pods/e8aa8388-e47a-4160-b614-81d17bede948/volumes" Sep 08 10:11:56.303777 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:11:56.303752 2568 kubelet.go:1648] "Image garbage collection succeeded" Sep 08 10:13:46.516820 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:13:46.516791 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-794745947f-qkcfq"] Sep 08 10:13:46.517206 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:13:46.517091 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8aa8388-e47a-4160-b614-81d17bede948" containerName="console" Sep 08 10:13:46.517206 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:13:46.517101 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8aa8388-e47a-4160-b614-81d17bede948" containerName="console" Sep 08 10:13:46.517206 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:13:46.517158 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8aa8388-e47a-4160-b614-81d17bede948" containerName="console" Sep 08 10:13:46.519928 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:13:46.519913 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-794745947f-qkcfq" Sep 08 10:13:46.522698 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:13:46.522679 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Sep 08 10:13:46.523620 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:13:46.523603 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Sep 08 10:13:46.523691 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:13:46.523603 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-5r86j\"" Sep 08 10:13:46.537355 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:13:46.537336 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-794745947f-qkcfq"] Sep 08 10:13:46.596996 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:13:46.596976 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsqrd\" (UniqueName: \"kubernetes.io/projected/3bcb6d04-6fcf-467f-ae66-48a518ebb7b2-kube-api-access-rsqrd\") pod \"feast-operator-controller-manager-794745947f-qkcfq\" (UID: \"3bcb6d04-6fcf-467f-ae66-48a518ebb7b2\") " pod="feast-operator-system/feast-operator-controller-manager-794745947f-qkcfq" Sep 08 10:13:46.698096 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:13:46.698071 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsqrd\" (UniqueName: \"kubernetes.io/projected/3bcb6d04-6fcf-467f-ae66-48a518ebb7b2-kube-api-access-rsqrd\") pod \"feast-operator-controller-manager-794745947f-qkcfq\" (UID: \"3bcb6d04-6fcf-467f-ae66-48a518ebb7b2\") " pod="feast-operator-system/feast-operator-controller-manager-794745947f-qkcfq" Sep 08 10:13:46.706718 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:13:46.706695 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsqrd\" (UniqueName: \"kubernetes.io/projected/3bcb6d04-6fcf-467f-ae66-48a518ebb7b2-kube-api-access-rsqrd\") pod \"feast-operator-controller-manager-794745947f-qkcfq\" (UID: \"3bcb6d04-6fcf-467f-ae66-48a518ebb7b2\") " pod="feast-operator-system/feast-operator-controller-manager-794745947f-qkcfq" Sep 08 10:13:46.829317 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:13:46.829249 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-794745947f-qkcfq" Sep 08 10:13:47.106480 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:13:47.106452 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-794745947f-qkcfq"] Sep 08 10:13:47.107951 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:13:47.107928 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bcb6d04_6fcf_467f_ae66_48a518ebb7b2.slice/crio-6ac95d2418ea7d2a776467b57168556607b33856074592c1cdb075fac831f805 WatchSource:0}: Error finding container 6ac95d2418ea7d2a776467b57168556607b33856074592c1cdb075fac831f805: Status 404 returned error can't find the container with id 6ac95d2418ea7d2a776467b57168556607b33856074592c1cdb075fac831f805 Sep 08 10:13:47.109812 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:13:47.109796 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 08 10:13:47.945833 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:13:47.945800 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-794745947f-qkcfq" event={"ID":"3bcb6d04-6fcf-467f-ae66-48a518ebb7b2","Type":"ContainerStarted","Data":"6ac95d2418ea7d2a776467b57168556607b33856074592c1cdb075fac831f805"} Sep 08 10:13:51.962597 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:13:51.962564 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-794745947f-qkcfq" event={"ID":"3bcb6d04-6fcf-467f-ae66-48a518ebb7b2","Type":"ContainerStarted","Data":"5d357a67c15a923906370a1d69d614218d8861e24d9bf6633de3193089ea251a"} Sep 08 10:13:51.962989 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:13:51.962675 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-794745947f-qkcfq" Sep 08 10:13:51.980502 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:13:51.980452 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-794745947f-qkcfq" podStartSLOduration=1.692569814 podStartE2EDuration="5.980438414s" totalImagesPullingTime="4.2878686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:13:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:13:51.979311776 +0000 UTC m=+416.023415855" watchObservedRunningTime="2026-09-08 10:13:51.980438414 +0000 UTC m=+416.024542482" Sep 08 10:14:02.968397 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:02.968371 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-794745947f-qkcfq" Sep 08 10:14:26.469072 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:26.468954 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652"] Sep 08 10:14:26.475604 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:26.475582 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:14:26.479116 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:26.479094 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 08 10:14:26.479313 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:26.479299 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 08 10:14:26.480243 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:26.479321 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-chwxz\"" Sep 08 10:14:26.480243 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:26.479694 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 08 10:14:26.480243 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:26.479700 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 08 10:14:26.480243 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:26.479812 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 08 10:14:26.480243 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:26.479949 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\"" Sep 08 10:14:26.480243 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:26.480063 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 08 10:14:26.489046 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:26.489025 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652"] Sep 08 10:14:26.586030 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:26.586002 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng75d\" (UniqueName: \"kubernetes.io/projected/ade31695-dfb6-42d3-a445-f740f334cfbf-kube-api-access-ng75d\") pod \"feast-simple-feast-setup-b5b55ccfc-6k652\" (UID: \"ade31695-dfb6-42d3-a445-f740f334cfbf\") " pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:14:26.586221 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:26.586206 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ade31695-dfb6-42d3-a445-f740f334cfbf-online-tls\") pod \"feast-simple-feast-setup-b5b55ccfc-6k652\" (UID: \"ade31695-dfb6-42d3-a445-f740f334cfbf\") " pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:14:26.586340 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:26.586326 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ade31695-dfb6-42d3-a445-f740f334cfbf-ui-tls\") pod \"feast-simple-feast-setup-b5b55ccfc-6k652\" (UID: \"ade31695-dfb6-42d3-a445-f740f334cfbf\") " pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:14:26.586493 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:26.586463 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ade31695-dfb6-42d3-a445-f740f334cfbf-offline-tls\") pod \"feast-simple-feast-setup-b5b55ccfc-6k652\" (UID: \"ade31695-dfb6-42d3-a445-f740f334cfbf\") " pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:14:26.586562 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:26.586522 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ade31695-dfb6-42d3-a445-f740f334cfbf-registry-tls\") pod \"feast-simple-feast-setup-b5b55ccfc-6k652\" (UID: \"ade31695-dfb6-42d3-a445-f740f334cfbf\") " pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:14:26.586616 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:26.586577 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ade31695-dfb6-42d3-a445-f740f334cfbf-feast-data\") pod \"feast-simple-feast-setup-b5b55ccfc-6k652\" (UID: \"ade31695-dfb6-42d3-a445-f740f334cfbf\") " pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:14:26.687065 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:26.687036 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ade31695-dfb6-42d3-a445-f740f334cfbf-online-tls\") pod \"feast-simple-feast-setup-b5b55ccfc-6k652\" (UID: \"ade31695-dfb6-42d3-a445-f740f334cfbf\") " pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:14:26.687176 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:26.687078 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ade31695-dfb6-42d3-a445-f740f334cfbf-ui-tls\") pod \"feast-simple-feast-setup-b5b55ccfc-6k652\" (UID: \"ade31695-dfb6-42d3-a445-f740f334cfbf\") " pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:14:26.687176 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:26.687106 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ade31695-dfb6-42d3-a445-f740f334cfbf-offline-tls\") pod \"feast-simple-feast-setup-b5b55ccfc-6k652\" (UID: \"ade31695-dfb6-42d3-a445-f740f334cfbf\") " pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:14:26.687288 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:14:26.687199 2568 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Sep 08 10:14:26.687288 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:14:26.687272 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ade31695-dfb6-42d3-a445-f740f334cfbf-ui-tls podName:ade31695-dfb6-42d3-a445-f740f334cfbf nodeName:}" failed. No retries permitted until 2026-09-08 10:14:27.187250245 +0000 UTC m=+451.231354304 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/ade31695-dfb6-42d3-a445-f740f334cfbf-ui-tls") pod "feast-simple-feast-setup-b5b55ccfc-6k652" (UID: "ade31695-dfb6-42d3-a445-f740f334cfbf") : secret "feast-simple-feast-setup-ui-tls" not found Sep 08 10:14:26.687395 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:26.687297 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ade31695-dfb6-42d3-a445-f740f334cfbf-registry-tls\") pod \"feast-simple-feast-setup-b5b55ccfc-6k652\" (UID: \"ade31695-dfb6-42d3-a445-f740f334cfbf\") " pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:14:26.687395 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:26.687360 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ade31695-dfb6-42d3-a445-f740f334cfbf-feast-data\") pod \"feast-simple-feast-setup-b5b55ccfc-6k652\" (UID: \"ade31695-dfb6-42d3-a445-f740f334cfbf\") " pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:14:26.687543 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:26.687398 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ng75d\" (UniqueName: \"kubernetes.io/projected/ade31695-dfb6-42d3-a445-f740f334cfbf-kube-api-access-ng75d\") pod \"feast-simple-feast-setup-b5b55ccfc-6k652\" (UID: \"ade31695-dfb6-42d3-a445-f740f334cfbf\") " pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:14:26.687543 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:14:26.687465 2568 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Sep 08 10:14:26.687543 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:14:26.687533 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ade31695-dfb6-42d3-a445-f740f334cfbf-registry-tls podName:ade31695-dfb6-42d3-a445-f740f334cfbf nodeName:}" failed. No retries permitted until 2026-09-08 10:14:27.187513904 +0000 UTC m=+451.231617963 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/ade31695-dfb6-42d3-a445-f740f334cfbf-registry-tls") pod "feast-simple-feast-setup-b5b55ccfc-6k652" (UID: "ade31695-dfb6-42d3-a445-f740f334cfbf") : secret "feast-simple-feast-setup-registry-tls" not found Sep 08 10:14:26.687732 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:26.687712 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ade31695-dfb6-42d3-a445-f740f334cfbf-feast-data\") pod \"feast-simple-feast-setup-b5b55ccfc-6k652\" (UID: \"ade31695-dfb6-42d3-a445-f740f334cfbf\") " pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:14:26.689780 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:26.689752 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ade31695-dfb6-42d3-a445-f740f334cfbf-offline-tls\") pod \"feast-simple-feast-setup-b5b55ccfc-6k652\" (UID: \"ade31695-dfb6-42d3-a445-f740f334cfbf\") " pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:14:26.689849 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:26.689791 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ade31695-dfb6-42d3-a445-f740f334cfbf-online-tls\") pod \"feast-simple-feast-setup-b5b55ccfc-6k652\" (UID: \"ade31695-dfb6-42d3-a445-f740f334cfbf\") " pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:14:26.701087 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:26.701061 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng75d\" (UniqueName: \"kubernetes.io/projected/ade31695-dfb6-42d3-a445-f740f334cfbf-kube-api-access-ng75d\") pod \"feast-simple-feast-setup-b5b55ccfc-6k652\" (UID: \"ade31695-dfb6-42d3-a445-f740f334cfbf\") " pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:14:27.191678 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:27.191649 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ade31695-dfb6-42d3-a445-f740f334cfbf-registry-tls\") pod \"feast-simple-feast-setup-b5b55ccfc-6k652\" (UID: \"ade31695-dfb6-42d3-a445-f740f334cfbf\") " pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:14:27.191836 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:27.191722 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ade31695-dfb6-42d3-a445-f740f334cfbf-ui-tls\") pod \"feast-simple-feast-setup-b5b55ccfc-6k652\" (UID: \"ade31695-dfb6-42d3-a445-f740f334cfbf\") " pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:14:27.194275 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:27.194250 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ade31695-dfb6-42d3-a445-f740f334cfbf-registry-tls\") pod \"feast-simple-feast-setup-b5b55ccfc-6k652\" (UID: \"ade31695-dfb6-42d3-a445-f740f334cfbf\") " pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:14:27.194387 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:27.194285 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ade31695-dfb6-42d3-a445-f740f334cfbf-ui-tls\") pod \"feast-simple-feast-setup-b5b55ccfc-6k652\" (UID: \"ade31695-dfb6-42d3-a445-f740f334cfbf\") " pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:14:27.398205 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:27.398180 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:14:27.675088 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:27.675067 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652"] Sep 08 10:14:27.676673 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:14:27.676646 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podade31695_dfb6_42d3_a445_f740f334cfbf.slice/crio-79db38060c493674a6b32ae29ec7c7c013b3c7996bb8a7bf3eb16c30c96130ad WatchSource:0}: Error finding container 79db38060c493674a6b32ae29ec7c7c013b3c7996bb8a7bf3eb16c30c96130ad: Status 404 returned error can't find the container with id 79db38060c493674a6b32ae29ec7c7c013b3c7996bb8a7bf3eb16c30c96130ad Sep 08 10:14:28.098647 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:28.098613 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" event={"ID":"ade31695-dfb6-42d3-a445-f740f334cfbf","Type":"ContainerStarted","Data":"79db38060c493674a6b32ae29ec7c7c013b3c7996bb8a7bf3eb16c30c96130ad"} Sep 08 10:14:49.902367 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:49.902312 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d6848b855-nkqs6"] Sep 08 10:14:54.213968 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:54.213934 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" event={"ID":"ade31695-dfb6-42d3-a445-f740f334cfbf","Type":"ContainerStarted","Data":"2367c24f2ad7d715ba5bd59525a1f17bce705b49d2b7d4775b2beb6cb92a9c82"} Sep 08 10:14:58.231133 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:58.231049 2568 generic.go:356] "Generic (PLEG): container finished" podID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerID="2367c24f2ad7d715ba5bd59525a1f17bce705b49d2b7d4775b2beb6cb92a9c82" exitCode=0 Sep 08 10:14:58.231488 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:58.231126 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" event={"ID":"ade31695-dfb6-42d3-a445-f740f334cfbf","Type":"ContainerDied","Data":"2367c24f2ad7d715ba5bd59525a1f17bce705b49d2b7d4775b2beb6cb92a9c82"} Sep 08 10:14:59.238205 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:14:59.238172 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" event={"ID":"ade31695-dfb6-42d3-a445-f740f334cfbf","Type":"ContainerStarted","Data":"75c79705fecec64aa74d31adb16d4f3ea5f120de557fd6041d2c5864ea70dea6"} Sep 08 10:15:04.260500 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:04.260434 2568 generic.go:356] "Generic (PLEG): container finished" podID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerID="75c79705fecec64aa74d31adb16d4f3ea5f120de557fd6041d2c5864ea70dea6" exitCode=0 Sep 08 10:15:04.260814 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:04.260507 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" event={"ID":"ade31695-dfb6-42d3-a445-f740f334cfbf","Type":"ContainerDied","Data":"75c79705fecec64aa74d31adb16d4f3ea5f120de557fd6041d2c5864ea70dea6"} Sep 08 10:15:04.269446 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:04.269430 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\"" Sep 08 10:15:05.270457 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:05.270404 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" event={"ID":"ade31695-dfb6-42d3-a445-f740f334cfbf","Type":"ContainerStarted","Data":"4f936f6327b419a195784884b74d37b1744f0c3a2b6f289108f42b83f291b637"} Sep 08 10:15:05.270942 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:05.270478 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" event={"ID":"ade31695-dfb6-42d3-a445-f740f334cfbf","Type":"ContainerStarted","Data":"1b85956983cff539d1185df4c76565435b02a60c74c8de88158c405157d5db4e"} Sep 08 10:15:05.270942 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:05.270500 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" event={"ID":"ade31695-dfb6-42d3-a445-f740f334cfbf","Type":"ContainerStarted","Data":"2a01fbfc52b842470e874f0f114f8703e68b7b8855fa39288e3d9de184bd012b"} Sep 08 10:15:05.270942 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:05.270519 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" event={"ID":"ade31695-dfb6-42d3-a445-f740f334cfbf","Type":"ContainerStarted","Data":"1a809c35958e13eb1781cbcaa974362d4bbf71148bdd1880e91e479da3307a7e"} Sep 08 10:15:05.297933 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:05.297862 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" podStartSLOduration=13.725892767 podStartE2EDuration="39.297841567s" totalImagesPullingTime="25.5719488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:14:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:15:05.294684127 +0000 UTC m=+489.338788197" watchObservedRunningTime="2026-09-08 10:15:05.297841567 +0000 UTC m=+489.341945637" Sep 08 10:15:06.401110 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:06.400824 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" podUID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused" Sep 08 10:15:06.401110 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:06.400922 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" podUID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.25:8816: connect: connection refused" Sep 08 10:15:06.401110 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:06.401009 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" podUID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.25:6571: connect: connection refused" Sep 08 10:15:06.401726 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:06.401168 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" podUID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused" Sep 08 10:15:06.405890 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:06.405864 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:15:06.406072 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:06.405910 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:15:06.406072 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:06.405929 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:15:06.406072 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:06.405946 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:15:07.398834 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:07.398796 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:15:07.399023 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:07.398854 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:15:07.399023 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:07.398877 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:15:07.399023 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:07.398897 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:15:09.398854 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:09.398738 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" podUID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.25:6571: connect: connection refused" Sep 08 10:15:09.398854 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:09.398769 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" podUID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused" Sep 08 10:15:09.398854 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:09.398781 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" podUID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.25:8816: connect: connection refused" Sep 08 10:15:09.398854 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:09.398804 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" podUID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused" Sep 08 10:15:12.398512 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:12.398452 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" podUID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused" Sep 08 10:15:12.405703 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:12.405676 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:15:12.405862 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:12.405815 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:15:12.405971 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:12.405957 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:15:12.406792 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:12.406769 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:15:12.406965 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:12.406912 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:15:12.409691 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:12.409670 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:15:14.931159 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:14.931100 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7d6848b855-nkqs6" podUID="beda3813-0e7a-4cbc-9d03-2dcc71cb867c" containerName="console" containerID="cri-o://15343b4770e3c896876d92ea61eb0f7154a01cb9bc2755f919020af9daa6aab6" gracePeriod=15 Sep 08 10:15:15.241398 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:15.241369 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d6848b855-nkqs6" Sep 08 10:15:15.323979 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:15.323942 2568 generic.go:356] "Generic (PLEG): container finished" podID="beda3813-0e7a-4cbc-9d03-2dcc71cb867c" containerID="15343b4770e3c896876d92ea61eb0f7154a01cb9bc2755f919020af9daa6aab6" exitCode=0 Sep 08 10:15:15.324136 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:15.323986 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d6848b855-nkqs6" Sep 08 10:15:15.324136 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:15.324025 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d6848b855-nkqs6" event={"ID":"beda3813-0e7a-4cbc-9d03-2dcc71cb867c","Type":"ContainerDied","Data":"15343b4770e3c896876d92ea61eb0f7154a01cb9bc2755f919020af9daa6aab6"} Sep 08 10:15:15.324136 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:15.324079 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d6848b855-nkqs6" event={"ID":"beda3813-0e7a-4cbc-9d03-2dcc71cb867c","Type":"ContainerDied","Data":"c360128cdd4047f95dea9d7ccee16a604a3881f140c545729c297df873e4ea5a"} Sep 08 10:15:15.324136 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:15.324100 2568 scope.go:117] "RemoveContainer" containerID="15343b4770e3c896876d92ea61eb0f7154a01cb9bc2755f919020af9daa6aab6" Sep 08 10:15:15.332309 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:15.332291 2568 scope.go:117] "RemoveContainer" containerID="15343b4770e3c896876d92ea61eb0f7154a01cb9bc2755f919020af9daa6aab6" Sep 08 10:15:15.332624 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:15:15.332607 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15343b4770e3c896876d92ea61eb0f7154a01cb9bc2755f919020af9daa6aab6\": container with ID starting with 15343b4770e3c896876d92ea61eb0f7154a01cb9bc2755f919020af9daa6aab6 not found: ID does not exist" containerID="15343b4770e3c896876d92ea61eb0f7154a01cb9bc2755f919020af9daa6aab6" Sep 08 10:15:15.332689 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:15.332634 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15343b4770e3c896876d92ea61eb0f7154a01cb9bc2755f919020af9daa6aab6"} err="failed to get container status \"15343b4770e3c896876d92ea61eb0f7154a01cb9bc2755f919020af9daa6aab6\": rpc error: code = NotFound desc = could not find container \"15343b4770e3c896876d92ea61eb0f7154a01cb9bc2755f919020af9daa6aab6\": container with ID starting with 15343b4770e3c896876d92ea61eb0f7154a01cb9bc2755f919020af9daa6aab6 not found: ID does not exist" Sep 08 10:15:15.392385 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:15.392364 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvwmn\" (UniqueName: \"kubernetes.io/projected/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-kube-api-access-nvwmn\") pod \"beda3813-0e7a-4cbc-9d03-2dcc71cb867c\" (UID: \"beda3813-0e7a-4cbc-9d03-2dcc71cb867c\") " Sep 08 10:15:15.392497 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:15.392464 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-oauth-serving-cert\") pod \"beda3813-0e7a-4cbc-9d03-2dcc71cb867c\" (UID: \"beda3813-0e7a-4cbc-9d03-2dcc71cb867c\") " Sep 08 10:15:15.392497 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:15.392488 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-console-serving-cert\") pod \"beda3813-0e7a-4cbc-9d03-2dcc71cb867c\" (UID: \"beda3813-0e7a-4cbc-9d03-2dcc71cb867c\") " Sep 08 10:15:15.392614 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:15.392515 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-console-config\") pod \"beda3813-0e7a-4cbc-9d03-2dcc71cb867c\" (UID: \"beda3813-0e7a-4cbc-9d03-2dcc71cb867c\") " Sep 08 10:15:15.392614 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:15.392546 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-console-oauth-config\") pod \"beda3813-0e7a-4cbc-9d03-2dcc71cb867c\" (UID: \"beda3813-0e7a-4cbc-9d03-2dcc71cb867c\") " Sep 08 10:15:15.392614 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:15.392585 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-trusted-ca-bundle\") pod \"beda3813-0e7a-4cbc-9d03-2dcc71cb867c\" (UID: \"beda3813-0e7a-4cbc-9d03-2dcc71cb867c\") " Sep 08 10:15:15.392614 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:15.392605 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-service-ca\") pod \"beda3813-0e7a-4cbc-9d03-2dcc71cb867c\" (UID: \"beda3813-0e7a-4cbc-9d03-2dcc71cb867c\") " Sep 08 10:15:15.392887 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:15.392863 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "beda3813-0e7a-4cbc-9d03-2dcc71cb867c" (UID: "beda3813-0e7a-4cbc-9d03-2dcc71cb867c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:15:15.392953 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:15.392926 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-console-config" (OuterVolumeSpecName: "console-config") pod "beda3813-0e7a-4cbc-9d03-2dcc71cb867c" (UID: "beda3813-0e7a-4cbc-9d03-2dcc71cb867c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:15:15.393077 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:15.393050 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "beda3813-0e7a-4cbc-9d03-2dcc71cb867c" (UID: "beda3813-0e7a-4cbc-9d03-2dcc71cb867c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:15:15.393318 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:15.393297 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-service-ca" (OuterVolumeSpecName: "service-ca") pod "beda3813-0e7a-4cbc-9d03-2dcc71cb867c" (UID: "beda3813-0e7a-4cbc-9d03-2dcc71cb867c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:15:15.394863 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:15.394839 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "beda3813-0e7a-4cbc-9d03-2dcc71cb867c" (UID: "beda3813-0e7a-4cbc-9d03-2dcc71cb867c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:15:15.394961 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:15.394929 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-kube-api-access-nvwmn" (OuterVolumeSpecName: "kube-api-access-nvwmn") pod "beda3813-0e7a-4cbc-9d03-2dcc71cb867c" (UID: "beda3813-0e7a-4cbc-9d03-2dcc71cb867c"). InnerVolumeSpecName "kube-api-access-nvwmn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 10:15:15.395117 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:15.395096 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "beda3813-0e7a-4cbc-9d03-2dcc71cb867c" (UID: "beda3813-0e7a-4cbc-9d03-2dcc71cb867c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:15:15.400074 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:15.400059 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:15:15.400898 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:15.400868 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:15:15.494180 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:15.494053 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-console-serving-cert\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:15:15.494329 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:15.494248 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-console-config\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:15:15.494329 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:15.494278 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-console-oauth-config\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:15:15.494329 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:15.494291 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-trusted-ca-bundle\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:15:15.494329 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:15.494301 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-service-ca\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:15:15.494329 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:15.494309 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvwmn\" (UniqueName: \"kubernetes.io/projected/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-kube-api-access-nvwmn\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:15:15.494329 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:15.494318 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/beda3813-0e7a-4cbc-9d03-2dcc71cb867c-oauth-serving-cert\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:15:15.645331 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:15.645305 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d6848b855-nkqs6"] Sep 08 10:15:15.652823 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:15.652800 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7d6848b855-nkqs6"] Sep 08 10:15:16.403196 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:15:16.403168 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="beda3813-0e7a-4cbc-9d03-2dcc71cb867c" path="/var/lib/kubelet/pods/beda3813-0e7a-4cbc-9d03-2dcc71cb867c/volumes" Sep 08 10:16:16.724675 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:16.724637 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652"] Sep 08 10:16:16.725237 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:16.724911 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" podUID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerName="registry" containerID="cri-o://1a809c35958e13eb1781cbcaa974362d4bbf71148bdd1880e91e479da3307a7e" gracePeriod=30 Sep 08 10:16:16.725237 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:16.724975 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" podUID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerName="online" containerID="cri-o://2a01fbfc52b842470e874f0f114f8703e68b7b8855fa39288e3d9de184bd012b" gracePeriod=30 Sep 08 10:16:16.725237 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:16.724968 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" podUID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerName="offline" containerID="cri-o://1b85956983cff539d1185df4c76565435b02a60c74c8de88158c405157d5db4e" gracePeriod=30 Sep 08 10:16:16.725237 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:16.725108 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" podUID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerName="ui" containerID="cri-o://4f936f6327b419a195784884b74d37b1744f0c3a2b6f289108f42b83f291b637" gracePeriod=30 Sep 08 10:16:17.279472 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:17.279437 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx"] Sep 08 10:16:17.280128 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:17.280103 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="beda3813-0e7a-4cbc-9d03-2dcc71cb867c" containerName="console" Sep 08 10:16:17.280128 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:17.280127 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="beda3813-0e7a-4cbc-9d03-2dcc71cb867c" containerName="console" Sep 08 10:16:17.280288 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:17.280213 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="beda3813-0e7a-4cbc-9d03-2dcc71cb867c" containerName="console" Sep 08 10:16:17.283666 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:17.283648 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:16:17.287647 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:17.287626 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-tsr2h\"" Sep 08 10:16:17.318112 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:17.318078 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx"] Sep 08 10:16:17.319240 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:17.319217 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e256fa60-9cb7-4570-958c-63e41d145e5f-feast-data\") pod \"feast-simple-feast-setup-b5b55ccfc-ww8zx\" (UID: \"e256fa60-9cb7-4570-958c-63e41d145e5f\") " pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:16:17.319356 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:17.319260 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e256fa60-9cb7-4570-958c-63e41d145e5f-offline-tls\") pod \"feast-simple-feast-setup-b5b55ccfc-ww8zx\" (UID: \"e256fa60-9cb7-4570-958c-63e41d145e5f\") " pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:16:17.319356 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:17.319288 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e256fa60-9cb7-4570-958c-63e41d145e5f-registry-tls\") pod \"feast-simple-feast-setup-b5b55ccfc-ww8zx\" (UID: \"e256fa60-9cb7-4570-958c-63e41d145e5f\") " pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:16:17.319356 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:17.319341 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5v4t\" (UniqueName: \"kubernetes.io/projected/e256fa60-9cb7-4570-958c-63e41d145e5f-kube-api-access-j5v4t\") pod \"feast-simple-feast-setup-b5b55ccfc-ww8zx\" (UID: \"e256fa60-9cb7-4570-958c-63e41d145e5f\") " pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:16:17.319481 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:17.319426 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e256fa60-9cb7-4570-958c-63e41d145e5f-online-tls\") pod \"feast-simple-feast-setup-b5b55ccfc-ww8zx\" (UID: \"e256fa60-9cb7-4570-958c-63e41d145e5f\") " pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:16:17.319522 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:17.319484 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e256fa60-9cb7-4570-958c-63e41d145e5f-ui-tls\") pod \"feast-simple-feast-setup-b5b55ccfc-ww8zx\" (UID: \"e256fa60-9cb7-4570-958c-63e41d145e5f\") " pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:16:17.420403 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:17.420371 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e256fa60-9cb7-4570-958c-63e41d145e5f-feast-data\") pod \"feast-simple-feast-setup-b5b55ccfc-ww8zx\" (UID: \"e256fa60-9cb7-4570-958c-63e41d145e5f\") " pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:16:17.420592 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:17.420438 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e256fa60-9cb7-4570-958c-63e41d145e5f-offline-tls\") pod \"feast-simple-feast-setup-b5b55ccfc-ww8zx\" (UID: \"e256fa60-9cb7-4570-958c-63e41d145e5f\") " pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:16:17.420592 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:17.420475 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e256fa60-9cb7-4570-958c-63e41d145e5f-registry-tls\") pod \"feast-simple-feast-setup-b5b55ccfc-ww8zx\" (UID: \"e256fa60-9cb7-4570-958c-63e41d145e5f\") " pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:16:17.420592 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:17.420501 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5v4t\" (UniqueName: \"kubernetes.io/projected/e256fa60-9cb7-4570-958c-63e41d145e5f-kube-api-access-j5v4t\") pod \"feast-simple-feast-setup-b5b55ccfc-ww8zx\" (UID: \"e256fa60-9cb7-4570-958c-63e41d145e5f\") " pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:16:17.420592 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:17.420535 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e256fa60-9cb7-4570-958c-63e41d145e5f-online-tls\") pod \"feast-simple-feast-setup-b5b55ccfc-ww8zx\" (UID: \"e256fa60-9cb7-4570-958c-63e41d145e5f\") " pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:16:17.421061 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:17.421026 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e256fa60-9cb7-4570-958c-63e41d145e5f-ui-tls\") pod \"feast-simple-feast-setup-b5b55ccfc-ww8zx\" (UID: \"e256fa60-9cb7-4570-958c-63e41d145e5f\") " pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:16:17.422531 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:17.421970 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e256fa60-9cb7-4570-958c-63e41d145e5f-feast-data\") pod \"feast-simple-feast-setup-b5b55ccfc-ww8zx\" (UID: \"e256fa60-9cb7-4570-958c-63e41d145e5f\") " pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:16:17.429857 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:17.429833 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e256fa60-9cb7-4570-958c-63e41d145e5f-registry-tls\") pod \"feast-simple-feast-setup-b5b55ccfc-ww8zx\" (UID: \"e256fa60-9cb7-4570-958c-63e41d145e5f\") " pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:16:17.432601 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:17.432553 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e256fa60-9cb7-4570-958c-63e41d145e5f-ui-tls\") pod \"feast-simple-feast-setup-b5b55ccfc-ww8zx\" (UID: \"e256fa60-9cb7-4570-958c-63e41d145e5f\") " pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:16:17.432751 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:17.432718 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e256fa60-9cb7-4570-958c-63e41d145e5f-online-tls\") pod \"feast-simple-feast-setup-b5b55ccfc-ww8zx\" (UID: \"e256fa60-9cb7-4570-958c-63e41d145e5f\") " pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:16:17.433300 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:17.432945 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e256fa60-9cb7-4570-958c-63e41d145e5f-offline-tls\") pod \"feast-simple-feast-setup-b5b55ccfc-ww8zx\" (UID: \"e256fa60-9cb7-4570-958c-63e41d145e5f\") " pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:16:17.433429 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:17.433366 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5v4t\" (UniqueName: \"kubernetes.io/projected/e256fa60-9cb7-4570-958c-63e41d145e5f-kube-api-access-j5v4t\") pod \"feast-simple-feast-setup-b5b55ccfc-ww8zx\" (UID: \"e256fa60-9cb7-4570-958c-63e41d145e5f\") " pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:16:17.557216 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:17.557188 2568 generic.go:356] "Generic (PLEG): container finished" podID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerID="4f936f6327b419a195784884b74d37b1744f0c3a2b6f289108f42b83f291b637" exitCode=0 Sep 08 10:16:17.557216 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:17.557215 2568 generic.go:356] "Generic (PLEG): container finished" podID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerID="2a01fbfc52b842470e874f0f114f8703e68b7b8855fa39288e3d9de184bd012b" exitCode=0 Sep 08 10:16:17.557341 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:17.557259 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" event={"ID":"ade31695-dfb6-42d3-a445-f740f334cfbf","Type":"ContainerDied","Data":"4f936f6327b419a195784884b74d37b1744f0c3a2b6f289108f42b83f291b637"} Sep 08 10:16:17.557341 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:17.557303 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" event={"ID":"ade31695-dfb6-42d3-a445-f740f334cfbf","Type":"ContainerDied","Data":"2a01fbfc52b842470e874f0f114f8703e68b7b8855fa39288e3d9de184bd012b"} Sep 08 10:16:17.594561 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:17.594541 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:16:17.899199 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:17.899166 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx"] Sep 08 10:16:17.908329 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:16:17.908302 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode256fa60_9cb7_4570_958c_63e41d145e5f.slice/crio-c4cfe8880414f3a55b920f973950a6908a350103d12050d01b8adb0ebbb7497a WatchSource:0}: Error finding container c4cfe8880414f3a55b920f973950a6908a350103d12050d01b8adb0ebbb7497a: Status 404 returned error can't find the container with id c4cfe8880414f3a55b920f973950a6908a350103d12050d01b8adb0ebbb7497a Sep 08 10:16:18.563082 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:18.563049 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" event={"ID":"e256fa60-9cb7-4570-958c-63e41d145e5f","Type":"ContainerStarted","Data":"9d15458c9589b665025bff2cd24631b454b8613e0da957c273188dc21f272eef"} Sep 08 10:16:18.563082 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:18.563090 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" event={"ID":"e256fa60-9cb7-4570-958c-63e41d145e5f","Type":"ContainerStarted","Data":"c4cfe8880414f3a55b920f973950a6908a350103d12050d01b8adb0ebbb7497a"} Sep 08 10:16:22.406814 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:22.406776 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" podUID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused" Sep 08 10:16:23.588203 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:23.588171 2568 generic.go:356] "Generic (PLEG): container finished" podID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerID="9d15458c9589b665025bff2cd24631b454b8613e0da957c273188dc21f272eef" exitCode=0 Sep 08 10:16:23.588619 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:23.588253 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" event={"ID":"e256fa60-9cb7-4570-958c-63e41d145e5f","Type":"ContainerDied","Data":"9d15458c9589b665025bff2cd24631b454b8613e0da957c273188dc21f272eef"} Sep 08 10:16:24.594425 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:24.594365 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" event={"ID":"e256fa60-9cb7-4570-958c-63e41d145e5f","Type":"ContainerStarted","Data":"891b09eb803c6dce6022d83b53c65cd535ed65ed8588bfdcbb66ae65a7cdb568"} Sep 08 10:16:25.401568 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:25.401523 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" podUID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused" Sep 08 10:16:29.623890 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:29.623860 2568 generic.go:356] "Generic (PLEG): container finished" podID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerID="891b09eb803c6dce6022d83b53c65cd535ed65ed8588bfdcbb66ae65a7cdb568" exitCode=0 Sep 08 10:16:29.624344 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:29.623937 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" event={"ID":"e256fa60-9cb7-4570-958c-63e41d145e5f","Type":"ContainerDied","Data":"891b09eb803c6dce6022d83b53c65cd535ed65ed8588bfdcbb66ae65a7cdb568"} Sep 08 10:16:30.634272 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:30.634227 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" event={"ID":"e256fa60-9cb7-4570-958c-63e41d145e5f","Type":"ContainerStarted","Data":"6a9ebab4791a0c32232ff2fcf10b70be778e8b13249d09beeb98bd57c0324ec2"} Sep 08 10:16:30.634771 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:30.634289 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" event={"ID":"e256fa60-9cb7-4570-958c-63e41d145e5f","Type":"ContainerStarted","Data":"81375e4aae42145e0f0a7f560f7bafdf906f434b9bf3f284752c08a932e2a0cf"} Sep 08 10:16:30.634771 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:30.634312 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" event={"ID":"e256fa60-9cb7-4570-958c-63e41d145e5f","Type":"ContainerStarted","Data":"f25a60e09810c2da1aca63de2b1498ac4f5ae993241d8ccdc16e39e391a22fdb"} Sep 08 10:16:30.634771 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:30.634329 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" event={"ID":"e256fa60-9cb7-4570-958c-63e41d145e5f","Type":"ContainerStarted","Data":"d321f2eada533c356f96c7e928c6c8a3886b6a9a9fad7ea49c8d99c0cf131a40"} Sep 08 10:16:30.667042 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:30.666977 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" podStartSLOduration=13.666954423 podStartE2EDuration="13.666954423s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:16:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:16:30.665085613 +0000 UTC m=+574.709189707" watchObservedRunningTime="2026-09-08 10:16:30.666954423 +0000 UTC m=+574.711058492" Sep 08 10:16:32.406783 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:32.406596 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" podUID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused" Sep 08 10:16:32.594873 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:32.594830 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:16:32.595061 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:32.594887 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:16:32.595061 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:32.594910 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:16:32.595061 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:32.594927 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:16:32.597222 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:32.597050 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" podUID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.26:6571: connect: connection refused" Sep 08 10:16:32.597222 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:32.597077 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" podUID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused" Sep 08 10:16:32.597222 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:32.597091 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" podUID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.26:8816: connect: connection refused" Sep 08 10:16:32.597222 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:32.597161 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" podUID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused" Sep 08 10:16:35.400945 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:35.400902 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" podUID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused" Sep 08 10:16:35.595140 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:35.595092 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" podUID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.26:6571: connect: connection refused" Sep 08 10:16:35.595447 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:35.595099 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" podUID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.26:8816: connect: connection refused" Sep 08 10:16:35.595562 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:35.595092 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" podUID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused" Sep 08 10:16:35.595562 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:35.595146 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" podUID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused" Sep 08 10:16:37.595156 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:37.595094 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:16:37.595156 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:37.595173 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:16:37.595597 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:37.595187 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:16:37.595597 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:37.595198 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:16:38.595916 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:38.595884 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:16:38.596351 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:38.596156 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:16:38.596638 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:38.596618 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:16:38.596742 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:38.596696 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:16:38.596792 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:38.596751 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:16:38.601136 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:38.601119 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:16:38.676566 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:38.676535 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:16:38.679263 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:38.679242 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:16:42.406666 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:42.406624 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" podUID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused" Sep 08 10:16:42.407182 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:42.406751 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:16:45.400775 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:45.400740 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" podUID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused" Sep 08 10:16:45.401139 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:45.400836 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:16:47.371106 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.371085 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:16:47.468899 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.468876 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ng75d\" (UniqueName: \"kubernetes.io/projected/ade31695-dfb6-42d3-a445-f740f334cfbf-kube-api-access-ng75d\") pod \"ade31695-dfb6-42d3-a445-f740f334cfbf\" (UID: \"ade31695-dfb6-42d3-a445-f740f334cfbf\") " Sep 08 10:16:47.469042 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.468918 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ade31695-dfb6-42d3-a445-f740f334cfbf-offline-tls\") pod \"ade31695-dfb6-42d3-a445-f740f334cfbf\" (UID: \"ade31695-dfb6-42d3-a445-f740f334cfbf\") " Sep 08 10:16:47.469042 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.468944 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ade31695-dfb6-42d3-a445-f740f334cfbf-registry-tls\") pod \"ade31695-dfb6-42d3-a445-f740f334cfbf\" (UID: \"ade31695-dfb6-42d3-a445-f740f334cfbf\") " Sep 08 10:16:47.469042 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.468983 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ade31695-dfb6-42d3-a445-f740f334cfbf-feast-data\") pod \"ade31695-dfb6-42d3-a445-f740f334cfbf\" (UID: \"ade31695-dfb6-42d3-a445-f740f334cfbf\") " Sep 08 10:16:47.469042 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.469006 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ade31695-dfb6-42d3-a445-f740f334cfbf-ui-tls\") pod \"ade31695-dfb6-42d3-a445-f740f334cfbf\" (UID: \"ade31695-dfb6-42d3-a445-f740f334cfbf\") " Sep 08 10:16:47.469042 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.469033 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ade31695-dfb6-42d3-a445-f740f334cfbf-online-tls\") pod \"ade31695-dfb6-42d3-a445-f740f334cfbf\" (UID: \"ade31695-dfb6-42d3-a445-f740f334cfbf\") " Sep 08 10:16:47.469634 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.469589 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ade31695-dfb6-42d3-a445-f740f334cfbf-feast-data" (OuterVolumeSpecName: "feast-data") pod "ade31695-dfb6-42d3-a445-f740f334cfbf" (UID: "ade31695-dfb6-42d3-a445-f740f334cfbf"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 10:16:47.471319 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.471289 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ade31695-dfb6-42d3-a445-f740f334cfbf-online-tls" (OuterVolumeSpecName: "online-tls") pod "ade31695-dfb6-42d3-a445-f740f334cfbf" (UID: "ade31695-dfb6-42d3-a445-f740f334cfbf"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:16:47.471319 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.471302 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ade31695-dfb6-42d3-a445-f740f334cfbf-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "ade31695-dfb6-42d3-a445-f740f334cfbf" (UID: "ade31695-dfb6-42d3-a445-f740f334cfbf"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:16:47.471473 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.471340 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ade31695-dfb6-42d3-a445-f740f334cfbf-kube-api-access-ng75d" (OuterVolumeSpecName: "kube-api-access-ng75d") pod "ade31695-dfb6-42d3-a445-f740f334cfbf" (UID: "ade31695-dfb6-42d3-a445-f740f334cfbf"). InnerVolumeSpecName "kube-api-access-ng75d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 10:16:47.471562 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.471546 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ade31695-dfb6-42d3-a445-f740f334cfbf-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ade31695-dfb6-42d3-a445-f740f334cfbf" (UID: "ade31695-dfb6-42d3-a445-f740f334cfbf"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:16:47.471676 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.471649 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ade31695-dfb6-42d3-a445-f740f334cfbf-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "ade31695-dfb6-42d3-a445-f740f334cfbf" (UID: "ade31695-dfb6-42d3-a445-f740f334cfbf"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:16:47.570300 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.570246 2568 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ade31695-dfb6-42d3-a445-f740f334cfbf-feast-data\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:16:47.570300 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.570266 2568 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ade31695-dfb6-42d3-a445-f740f334cfbf-ui-tls\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:16:47.570300 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.570292 2568 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ade31695-dfb6-42d3-a445-f740f334cfbf-online-tls\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:16:47.570300 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.570300 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ng75d\" (UniqueName: \"kubernetes.io/projected/ade31695-dfb6-42d3-a445-f740f334cfbf-kube-api-access-ng75d\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:16:47.570524 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.570309 2568 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ade31695-dfb6-42d3-a445-f740f334cfbf-offline-tls\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:16:47.570524 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.570317 2568 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ade31695-dfb6-42d3-a445-f740f334cfbf-registry-tls\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:16:47.719513 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.719483 2568 generic.go:356] "Generic (PLEG): container finished" podID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerID="1b85956983cff539d1185df4c76565435b02a60c74c8de88158c405157d5db4e" exitCode=137 Sep 08 10:16:47.719513 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.719508 2568 generic.go:356] "Generic (PLEG): container finished" podID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerID="1a809c35958e13eb1781cbcaa974362d4bbf71148bdd1880e91e479da3307a7e" exitCode=137 Sep 08 10:16:47.719647 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.719540 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" Sep 08 10:16:47.719647 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.719560 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" event={"ID":"ade31695-dfb6-42d3-a445-f740f334cfbf","Type":"ContainerDied","Data":"1b85956983cff539d1185df4c76565435b02a60c74c8de88158c405157d5db4e"} Sep 08 10:16:47.719647 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.719604 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" event={"ID":"ade31695-dfb6-42d3-a445-f740f334cfbf","Type":"ContainerDied","Data":"1a809c35958e13eb1781cbcaa974362d4bbf71148bdd1880e91e479da3307a7e"} Sep 08 10:16:47.719647 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.719621 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652" event={"ID":"ade31695-dfb6-42d3-a445-f740f334cfbf","Type":"ContainerDied","Data":"79db38060c493674a6b32ae29ec7c7c013b3c7996bb8a7bf3eb16c30c96130ad"} Sep 08 10:16:47.719647 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.719637 2568 scope.go:117] "RemoveContainer" containerID="4f936f6327b419a195784884b74d37b1744f0c3a2b6f289108f42b83f291b637" Sep 08 10:16:47.727989 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.727974 2568 scope.go:117] "RemoveContainer" containerID="1b85956983cff539d1185df4c76565435b02a60c74c8de88158c405157d5db4e" Sep 08 10:16:47.739489 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.739474 2568 scope.go:117] "RemoveContainer" containerID="2a01fbfc52b842470e874f0f114f8703e68b7b8855fa39288e3d9de184bd012b" Sep 08 10:16:47.742180 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.742162 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652"] Sep 08 10:16:47.745621 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.745598 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-b5b55ccfc-6k652"] Sep 08 10:16:47.747196 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.747182 2568 scope.go:117] "RemoveContainer" containerID="1a809c35958e13eb1781cbcaa974362d4bbf71148bdd1880e91e479da3307a7e" Sep 08 10:16:47.754273 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.754258 2568 scope.go:117] "RemoveContainer" containerID="75c79705fecec64aa74d31adb16d4f3ea5f120de557fd6041d2c5864ea70dea6" Sep 08 10:16:47.761147 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.761134 2568 scope.go:117] "RemoveContainer" containerID="2367c24f2ad7d715ba5bd59525a1f17bce705b49d2b7d4775b2beb6cb92a9c82" Sep 08 10:16:47.767857 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.767843 2568 scope.go:117] "RemoveContainer" containerID="4f936f6327b419a195784884b74d37b1744f0c3a2b6f289108f42b83f291b637" Sep 08 10:16:47.768101 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:16:47.768082 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f936f6327b419a195784884b74d37b1744f0c3a2b6f289108f42b83f291b637\": container with ID starting with 4f936f6327b419a195784884b74d37b1744f0c3a2b6f289108f42b83f291b637 not found: ID does not exist" containerID="4f936f6327b419a195784884b74d37b1744f0c3a2b6f289108f42b83f291b637" Sep 08 10:16:47.768154 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.768110 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f936f6327b419a195784884b74d37b1744f0c3a2b6f289108f42b83f291b637"} err="failed to get container status \"4f936f6327b419a195784884b74d37b1744f0c3a2b6f289108f42b83f291b637\": rpc error: code = NotFound desc = could not find container \"4f936f6327b419a195784884b74d37b1744f0c3a2b6f289108f42b83f291b637\": container with ID starting with 4f936f6327b419a195784884b74d37b1744f0c3a2b6f289108f42b83f291b637 not found: ID does not exist" Sep 08 10:16:47.768154 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.768128 2568 scope.go:117] "RemoveContainer" containerID="1b85956983cff539d1185df4c76565435b02a60c74c8de88158c405157d5db4e" Sep 08 10:16:47.768389 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:16:47.768373 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b85956983cff539d1185df4c76565435b02a60c74c8de88158c405157d5db4e\": container with ID starting with 1b85956983cff539d1185df4c76565435b02a60c74c8de88158c405157d5db4e not found: ID does not exist" containerID="1b85956983cff539d1185df4c76565435b02a60c74c8de88158c405157d5db4e" Sep 08 10:16:47.768481 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.768391 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b85956983cff539d1185df4c76565435b02a60c74c8de88158c405157d5db4e"} err="failed to get container status \"1b85956983cff539d1185df4c76565435b02a60c74c8de88158c405157d5db4e\": rpc error: code = NotFound desc = could not find container \"1b85956983cff539d1185df4c76565435b02a60c74c8de88158c405157d5db4e\": container with ID starting with 1b85956983cff539d1185df4c76565435b02a60c74c8de88158c405157d5db4e not found: ID does not exist" Sep 08 10:16:47.768481 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.768404 2568 scope.go:117] "RemoveContainer" containerID="2a01fbfc52b842470e874f0f114f8703e68b7b8855fa39288e3d9de184bd012b" Sep 08 10:16:47.768711 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:16:47.768681 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a01fbfc52b842470e874f0f114f8703e68b7b8855fa39288e3d9de184bd012b\": container with ID starting with 2a01fbfc52b842470e874f0f114f8703e68b7b8855fa39288e3d9de184bd012b not found: ID does not exist" containerID="2a01fbfc52b842470e874f0f114f8703e68b7b8855fa39288e3d9de184bd012b" Sep 08 10:16:47.768826 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.768715 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a01fbfc52b842470e874f0f114f8703e68b7b8855fa39288e3d9de184bd012b"} err="failed to get container status \"2a01fbfc52b842470e874f0f114f8703e68b7b8855fa39288e3d9de184bd012b\": rpc error: code = NotFound desc = could not find container \"2a01fbfc52b842470e874f0f114f8703e68b7b8855fa39288e3d9de184bd012b\": container with ID starting with 2a01fbfc52b842470e874f0f114f8703e68b7b8855fa39288e3d9de184bd012b not found: ID does not exist" Sep 08 10:16:47.768826 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.768731 2568 scope.go:117] "RemoveContainer" containerID="1a809c35958e13eb1781cbcaa974362d4bbf71148bdd1880e91e479da3307a7e" Sep 08 10:16:47.769121 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:16:47.769098 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a809c35958e13eb1781cbcaa974362d4bbf71148bdd1880e91e479da3307a7e\": container with ID starting with 1a809c35958e13eb1781cbcaa974362d4bbf71148bdd1880e91e479da3307a7e not found: ID does not exist" containerID="1a809c35958e13eb1781cbcaa974362d4bbf71148bdd1880e91e479da3307a7e" Sep 08 10:16:47.769194 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.769127 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a809c35958e13eb1781cbcaa974362d4bbf71148bdd1880e91e479da3307a7e"} err="failed to get container status \"1a809c35958e13eb1781cbcaa974362d4bbf71148bdd1880e91e479da3307a7e\": rpc error: code = NotFound desc = could not find container \"1a809c35958e13eb1781cbcaa974362d4bbf71148bdd1880e91e479da3307a7e\": container with ID starting with 1a809c35958e13eb1781cbcaa974362d4bbf71148bdd1880e91e479da3307a7e not found: ID does not exist" Sep 08 10:16:47.769194 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.769149 2568 scope.go:117] "RemoveContainer" containerID="75c79705fecec64aa74d31adb16d4f3ea5f120de557fd6041d2c5864ea70dea6" Sep 08 10:16:47.769461 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:16:47.769437 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75c79705fecec64aa74d31adb16d4f3ea5f120de557fd6041d2c5864ea70dea6\": container with ID starting with 75c79705fecec64aa74d31adb16d4f3ea5f120de557fd6041d2c5864ea70dea6 not found: ID does not exist" containerID="75c79705fecec64aa74d31adb16d4f3ea5f120de557fd6041d2c5864ea70dea6" Sep 08 10:16:47.769550 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.769468 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75c79705fecec64aa74d31adb16d4f3ea5f120de557fd6041d2c5864ea70dea6"} err="failed to get container status \"75c79705fecec64aa74d31adb16d4f3ea5f120de557fd6041d2c5864ea70dea6\": rpc error: code = NotFound desc = could not find container \"75c79705fecec64aa74d31adb16d4f3ea5f120de557fd6041d2c5864ea70dea6\": container with ID starting with 75c79705fecec64aa74d31adb16d4f3ea5f120de557fd6041d2c5864ea70dea6 not found: ID does not exist" Sep 08 10:16:47.769550 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.769486 2568 scope.go:117] "RemoveContainer" containerID="2367c24f2ad7d715ba5bd59525a1f17bce705b49d2b7d4775b2beb6cb92a9c82" Sep 08 10:16:47.769761 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:16:47.769742 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2367c24f2ad7d715ba5bd59525a1f17bce705b49d2b7d4775b2beb6cb92a9c82\": container with ID starting with 2367c24f2ad7d715ba5bd59525a1f17bce705b49d2b7d4775b2beb6cb92a9c82 not found: ID does not exist" containerID="2367c24f2ad7d715ba5bd59525a1f17bce705b49d2b7d4775b2beb6cb92a9c82" Sep 08 10:16:47.769825 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.769768 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2367c24f2ad7d715ba5bd59525a1f17bce705b49d2b7d4775b2beb6cb92a9c82"} err="failed to get container status \"2367c24f2ad7d715ba5bd59525a1f17bce705b49d2b7d4775b2beb6cb92a9c82\": rpc error: code = NotFound desc = could not find container \"2367c24f2ad7d715ba5bd59525a1f17bce705b49d2b7d4775b2beb6cb92a9c82\": container with ID starting with 2367c24f2ad7d715ba5bd59525a1f17bce705b49d2b7d4775b2beb6cb92a9c82 not found: ID does not exist" Sep 08 10:16:47.769825 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.769788 2568 scope.go:117] "RemoveContainer" containerID="4f936f6327b419a195784884b74d37b1744f0c3a2b6f289108f42b83f291b637" Sep 08 10:16:47.770004 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.769987 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f936f6327b419a195784884b74d37b1744f0c3a2b6f289108f42b83f291b637"} err="failed to get container status \"4f936f6327b419a195784884b74d37b1744f0c3a2b6f289108f42b83f291b637\": rpc error: code = NotFound desc = could not find container \"4f936f6327b419a195784884b74d37b1744f0c3a2b6f289108f42b83f291b637\": container with ID starting with 4f936f6327b419a195784884b74d37b1744f0c3a2b6f289108f42b83f291b637 not found: ID does not exist" Sep 08 10:16:47.770062 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.770005 2568 scope.go:117] "RemoveContainer" containerID="1b85956983cff539d1185df4c76565435b02a60c74c8de88158c405157d5db4e" Sep 08 10:16:47.770276 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.770254 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b85956983cff539d1185df4c76565435b02a60c74c8de88158c405157d5db4e"} err="failed to get container status \"1b85956983cff539d1185df4c76565435b02a60c74c8de88158c405157d5db4e\": rpc error: code = NotFound desc = could not find container \"1b85956983cff539d1185df4c76565435b02a60c74c8de88158c405157d5db4e\": container with ID starting with 1b85956983cff539d1185df4c76565435b02a60c74c8de88158c405157d5db4e not found: ID does not exist" Sep 08 10:16:47.770276 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.770274 2568 scope.go:117] "RemoveContainer" containerID="2a01fbfc52b842470e874f0f114f8703e68b7b8855fa39288e3d9de184bd012b" Sep 08 10:16:47.770529 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.770511 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a01fbfc52b842470e874f0f114f8703e68b7b8855fa39288e3d9de184bd012b"} err="failed to get container status \"2a01fbfc52b842470e874f0f114f8703e68b7b8855fa39288e3d9de184bd012b\": rpc error: code = NotFound desc = could not find container \"2a01fbfc52b842470e874f0f114f8703e68b7b8855fa39288e3d9de184bd012b\": container with ID starting with 2a01fbfc52b842470e874f0f114f8703e68b7b8855fa39288e3d9de184bd012b not found: ID does not exist" Sep 08 10:16:47.770579 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.770530 2568 scope.go:117] "RemoveContainer" containerID="1a809c35958e13eb1781cbcaa974362d4bbf71148bdd1880e91e479da3307a7e" Sep 08 10:16:47.770770 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.770753 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a809c35958e13eb1781cbcaa974362d4bbf71148bdd1880e91e479da3307a7e"} err="failed to get container status \"1a809c35958e13eb1781cbcaa974362d4bbf71148bdd1880e91e479da3307a7e\": rpc error: code = NotFound desc = could not find container \"1a809c35958e13eb1781cbcaa974362d4bbf71148bdd1880e91e479da3307a7e\": container with ID starting with 1a809c35958e13eb1781cbcaa974362d4bbf71148bdd1880e91e479da3307a7e not found: ID does not exist" Sep 08 10:16:47.770828 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.770771 2568 scope.go:117] "RemoveContainer" containerID="75c79705fecec64aa74d31adb16d4f3ea5f120de557fd6041d2c5864ea70dea6" Sep 08 10:16:47.771008 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.770992 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75c79705fecec64aa74d31adb16d4f3ea5f120de557fd6041d2c5864ea70dea6"} err="failed to get container status \"75c79705fecec64aa74d31adb16d4f3ea5f120de557fd6041d2c5864ea70dea6\": rpc error: code = NotFound desc = could not find container \"75c79705fecec64aa74d31adb16d4f3ea5f120de557fd6041d2c5864ea70dea6\": container with ID starting with 75c79705fecec64aa74d31adb16d4f3ea5f120de557fd6041d2c5864ea70dea6 not found: ID does not exist" Sep 08 10:16:47.771050 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.771010 2568 scope.go:117] "RemoveContainer" containerID="2367c24f2ad7d715ba5bd59525a1f17bce705b49d2b7d4775b2beb6cb92a9c82" Sep 08 10:16:47.771229 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:47.771212 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2367c24f2ad7d715ba5bd59525a1f17bce705b49d2b7d4775b2beb6cb92a9c82"} err="failed to get container status \"2367c24f2ad7d715ba5bd59525a1f17bce705b49d2b7d4775b2beb6cb92a9c82\": rpc error: code = NotFound desc = could not find container \"2367c24f2ad7d715ba5bd59525a1f17bce705b49d2b7d4775b2beb6cb92a9c82\": container with ID starting with 2367c24f2ad7d715ba5bd59525a1f17bce705b49d2b7d4775b2beb6cb92a9c82 not found: ID does not exist" Sep 08 10:16:48.403173 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:16:48.403146 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ade31695-dfb6-42d3-a445-f740f334cfbf" path="/var/lib/kubelet/pods/ade31695-dfb6-42d3-a445-f740f334cfbf/volumes" Sep 08 10:17:42.351894 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.351855 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw"] Sep 08 10:17:42.352389 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.352172 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerName="online" Sep 08 10:17:42.352389 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.352187 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerName="online" Sep 08 10:17:42.352389 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.352198 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerName="ui" Sep 08 10:17:42.352389 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.352206 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerName="ui" Sep 08 10:17:42.352389 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.352220 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerName="offline" Sep 08 10:17:42.352389 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.352226 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerName="offline" Sep 08 10:17:42.352389 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.352233 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerName="feast-apply" Sep 08 10:17:42.352389 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.352237 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerName="feast-apply" Sep 08 10:17:42.352389 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.352250 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerName="registry" Sep 08 10:17:42.352389 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.352255 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerName="registry" Sep 08 10:17:42.352389 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.352263 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerName="feast-init" Sep 08 10:17:42.352389 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.352271 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerName="feast-init" Sep 08 10:17:42.352389 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.352337 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerName="ui" Sep 08 10:17:42.352389 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.352347 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerName="registry" Sep 08 10:17:42.352389 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.352356 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerName="offline" Sep 08 10:17:42.352389 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.352368 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="ade31695-dfb6-42d3-a445-f740f334cfbf" containerName="online" Sep 08 10:17:42.355598 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.355582 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" Sep 08 10:17:42.359527 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.359508 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 08 10:17:42.360105 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.360086 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 08 10:17:42.360271 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.360253 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 08 10:17:42.360455 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.360435 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-qv685\"" Sep 08 10:17:42.360566 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.360515 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 08 10:17:42.360652 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.360638 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 08 10:17:42.361877 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.361858 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 08 10:17:42.361976 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.361881 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\"" Sep 08 10:17:42.372022 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.371998 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw"] Sep 08 10:17:42.452437 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.452390 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-registry-tls\") pod \"feast-simple-feast-remote-setup-66966cfd48-w8vbw\" (UID: \"9ad9e974-8426-46fa-bb7b-32f3c6ca792c\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" Sep 08 10:17:42.452586 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.452444 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-ui-tls\") pod \"feast-simple-feast-remote-setup-66966cfd48-w8vbw\" (UID: \"9ad9e974-8426-46fa-bb7b-32f3c6ca792c\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" Sep 08 10:17:42.452586 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.452519 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hmmt\" (UniqueName: \"kubernetes.io/projected/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-kube-api-access-8hmmt\") pod \"feast-simple-feast-remote-setup-66966cfd48-w8vbw\" (UID: \"9ad9e974-8426-46fa-bb7b-32f3c6ca792c\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" Sep 08 10:17:42.452586 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.452565 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-offline-tls\") pod \"feast-simple-feast-remote-setup-66966cfd48-w8vbw\" (UID: \"9ad9e974-8426-46fa-bb7b-32f3c6ca792c\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" Sep 08 10:17:42.452745 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.452611 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-online-tls\") pod \"feast-simple-feast-remote-setup-66966cfd48-w8vbw\" (UID: \"9ad9e974-8426-46fa-bb7b-32f3c6ca792c\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" Sep 08 10:17:42.452745 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.452670 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-feast-data\") pod \"feast-simple-feast-remote-setup-66966cfd48-w8vbw\" (UID: \"9ad9e974-8426-46fa-bb7b-32f3c6ca792c\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" Sep 08 10:17:42.553569 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.553541 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-online-tls\") pod \"feast-simple-feast-remote-setup-66966cfd48-w8vbw\" (UID: \"9ad9e974-8426-46fa-bb7b-32f3c6ca792c\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" Sep 08 10:17:42.553701 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.553597 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-feast-data\") pod \"feast-simple-feast-remote-setup-66966cfd48-w8vbw\" (UID: \"9ad9e974-8426-46fa-bb7b-32f3c6ca792c\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" Sep 08 10:17:42.553701 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.553636 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-registry-tls\") pod \"feast-simple-feast-remote-setup-66966cfd48-w8vbw\" (UID: \"9ad9e974-8426-46fa-bb7b-32f3c6ca792c\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" Sep 08 10:17:42.553701 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.553673 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-ui-tls\") pod \"feast-simple-feast-remote-setup-66966cfd48-w8vbw\" (UID: \"9ad9e974-8426-46fa-bb7b-32f3c6ca792c\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" Sep 08 10:17:42.553843 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.553705 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hmmt\" (UniqueName: \"kubernetes.io/projected/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-kube-api-access-8hmmt\") pod \"feast-simple-feast-remote-setup-66966cfd48-w8vbw\" (UID: \"9ad9e974-8426-46fa-bb7b-32f3c6ca792c\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" Sep 08 10:17:42.553843 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.553735 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-offline-tls\") pod \"feast-simple-feast-remote-setup-66966cfd48-w8vbw\" (UID: \"9ad9e974-8426-46fa-bb7b-32f3c6ca792c\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" Sep 08 10:17:42.553943 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:17:42.553841 2568 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 08 10:17:42.553943 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:17:42.553917 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-ui-tls podName:9ad9e974-8426-46fa-bb7b-32f3c6ca792c nodeName:}" failed. No retries permitted until 2026-09-08 10:17:43.053894809 +0000 UTC m=+647.097998861 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-ui-tls") pod "feast-simple-feast-remote-setup-66966cfd48-w8vbw" (UID: "9ad9e974-8426-46fa-bb7b-32f3c6ca792c") : secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 08 10:17:42.554109 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.554086 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-feast-data\") pod \"feast-simple-feast-remote-setup-66966cfd48-w8vbw\" (UID: \"9ad9e974-8426-46fa-bb7b-32f3c6ca792c\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" Sep 08 10:17:42.554333 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.554314 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-registry-tls\") pod \"feast-simple-feast-remote-setup-66966cfd48-w8vbw\" (UID: \"9ad9e974-8426-46fa-bb7b-32f3c6ca792c\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" Sep 08 10:17:42.555960 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.555936 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-online-tls\") pod \"feast-simple-feast-remote-setup-66966cfd48-w8vbw\" (UID: \"9ad9e974-8426-46fa-bb7b-32f3c6ca792c\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" Sep 08 10:17:42.556249 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.556234 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-offline-tls\") pod \"feast-simple-feast-remote-setup-66966cfd48-w8vbw\" (UID: \"9ad9e974-8426-46fa-bb7b-32f3c6ca792c\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" Sep 08 10:17:42.565666 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:42.565646 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hmmt\" (UniqueName: \"kubernetes.io/projected/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-kube-api-access-8hmmt\") pod \"feast-simple-feast-remote-setup-66966cfd48-w8vbw\" (UID: \"9ad9e974-8426-46fa-bb7b-32f3c6ca792c\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" Sep 08 10:17:43.057397 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:43.057362 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-ui-tls\") pod \"feast-simple-feast-remote-setup-66966cfd48-w8vbw\" (UID: \"9ad9e974-8426-46fa-bb7b-32f3c6ca792c\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" Sep 08 10:17:43.059769 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:43.059741 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-ui-tls\") pod \"feast-simple-feast-remote-setup-66966cfd48-w8vbw\" (UID: \"9ad9e974-8426-46fa-bb7b-32f3c6ca792c\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" Sep 08 10:17:43.265942 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:43.265917 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" Sep 08 10:17:43.562566 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:43.562492 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw"] Sep 08 10:17:43.565018 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:17:43.564992 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9ad9e974_8426_46fa_bb7b_32f3c6ca792c.slice/crio-950516065dae8e0148798770f46be78a46715d459e2564c5822aa173875f4fa4 WatchSource:0}: Error finding container 950516065dae8e0148798770f46be78a46715d459e2564c5822aa173875f4fa4: Status 404 returned error can't find the container with id 950516065dae8e0148798770f46be78a46715d459e2564c5822aa173875f4fa4 Sep 08 10:17:43.934756 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:43.934716 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" event={"ID":"9ad9e974-8426-46fa-bb7b-32f3c6ca792c","Type":"ContainerStarted","Data":"5cf04d782734794a66f30faf6030480ca0aace1fcc44f8edfacd969b92dfeacc"} Sep 08 10:17:43.934756 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:43.934758 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" event={"ID":"9ad9e974-8426-46fa-bb7b-32f3c6ca792c","Type":"ContainerStarted","Data":"950516065dae8e0148798770f46be78a46715d459e2564c5822aa173875f4fa4"} Sep 08 10:17:48.959574 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:48.959543 2568 generic.go:356] "Generic (PLEG): container finished" podID="9ad9e974-8426-46fa-bb7b-32f3c6ca792c" containerID="5cf04d782734794a66f30faf6030480ca0aace1fcc44f8edfacd969b92dfeacc" exitCode=0 Sep 08 10:17:48.959933 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:48.959615 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" event={"ID":"9ad9e974-8426-46fa-bb7b-32f3c6ca792c","Type":"ContainerDied","Data":"5cf04d782734794a66f30faf6030480ca0aace1fcc44f8edfacd969b92dfeacc"} Sep 08 10:17:49.966588 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:17:49.966548 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" event={"ID":"9ad9e974-8426-46fa-bb7b-32f3c6ca792c","Type":"ContainerStarted","Data":"cfba0f518766fb89c67198e4e49a34bf4b1f3743d792db40503e6504831dd2fb"} Sep 08 10:18:17.076652 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:18:17.076621 2568 generic.go:356] "Generic (PLEG): container finished" podID="9ad9e974-8426-46fa-bb7b-32f3c6ca792c" containerID="cfba0f518766fb89c67198e4e49a34bf4b1f3743d792db40503e6504831dd2fb" exitCode=0 Sep 08 10:18:17.077069 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:18:17.076669 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" event={"ID":"9ad9e974-8426-46fa-bb7b-32f3c6ca792c","Type":"ContainerDied","Data":"cfba0f518766fb89c67198e4e49a34bf4b1f3743d792db40503e6504831dd2fb"} Sep 08 10:18:17.082487 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:18:17.082468 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\"" Sep 08 10:18:18.085328 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:18:18.085289 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" event={"ID":"9ad9e974-8426-46fa-bb7b-32f3c6ca792c","Type":"ContainerStarted","Data":"2a8f7dfefd4bedeaafd18b2e0b0cca9dd3f6b6ab9ac6a7c426e106fbe2a9ba0e"} Sep 08 10:18:18.085719 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:18:18.085352 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" event={"ID":"9ad9e974-8426-46fa-bb7b-32f3c6ca792c","Type":"ContainerStarted","Data":"0586eea4019b04ade9ef2595f6b200711b3821d00ef3a4be8a72013268f420a0"} Sep 08 10:18:18.085719 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:18:18.085375 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" event={"ID":"9ad9e974-8426-46fa-bb7b-32f3c6ca792c","Type":"ContainerStarted","Data":"d3f9dc7039916686ff266ce329fec1954b5350c0461b598782b0c8919dbccb4b"} Sep 08 10:18:18.119894 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:18:18.119839 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" podStartSLOduration=36.119819828 podStartE2EDuration="36.119819828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:17:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:18:18.119582716 +0000 UTC m=+682.163686815" watchObservedRunningTime="2026-09-08 10:18:18.119819828 +0000 UTC m=+682.163923899" Sep 08 10:18:19.266769 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:18:19.266730 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" Sep 08 10:18:19.267307 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:18:19.266793 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" Sep 08 10:18:19.267307 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:18:19.266814 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" Sep 08 10:18:19.268711 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:18:19.268671 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" podUID="9ad9e974-8426-46fa-bb7b-32f3c6ca792c" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.27:8443: connect: connection refused" Sep 08 10:18:19.268711 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:18:19.268695 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" podUID="9ad9e974-8426-46fa-bb7b-32f3c6ca792c" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.27:8816: connect: connection refused" Sep 08 10:18:19.268910 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:18:19.268718 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" podUID="9ad9e974-8426-46fa-bb7b-32f3c6ca792c" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused" Sep 08 10:18:22.266541 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:18:22.266496 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" podUID="9ad9e974-8426-46fa-bb7b-32f3c6ca792c" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.27:8816: connect: connection refused" Sep 08 10:18:22.267015 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:18:22.266500 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" podUID="9ad9e974-8426-46fa-bb7b-32f3c6ca792c" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused" Sep 08 10:18:22.267015 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:18:22.266510 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" podUID="9ad9e974-8426-46fa-bb7b-32f3c6ca792c" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.27:8443: connect: connection refused" Sep 08 10:18:23.266528 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:18:23.266496 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" Sep 08 10:18:23.266711 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:18:23.266553 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" Sep 08 10:18:23.266711 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:18:23.266571 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" Sep 08 10:18:25.267405 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:18:25.267378 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" Sep 08 10:18:25.267883 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:18:25.267499 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" Sep 08 10:18:25.268087 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:18:25.268067 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" Sep 08 10:18:25.268386 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:18:25.268369 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" Sep 08 10:18:25.298544 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:18:25.298519 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" Sep 08 10:18:26.153948 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:18:26.153923 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" Sep 08 10:19:28.751338 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:28.751306 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw"] Sep 08 10:19:28.751864 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:28.751585 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" podUID="9ad9e974-8426-46fa-bb7b-32f3c6ca792c" containerName="online" containerID="cri-o://d3f9dc7039916686ff266ce329fec1954b5350c0461b598782b0c8919dbccb4b" gracePeriod=30 Sep 08 10:19:28.751864 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:28.751622 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" podUID="9ad9e974-8426-46fa-bb7b-32f3c6ca792c" containerName="ui" containerID="cri-o://2a8f7dfefd4bedeaafd18b2e0b0cca9dd3f6b6ab9ac6a7c426e106fbe2a9ba0e" gracePeriod=30 Sep 08 10:19:28.751864 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:28.751620 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" podUID="9ad9e974-8426-46fa-bb7b-32f3c6ca792c" containerName="offline" containerID="cri-o://0586eea4019b04ade9ef2595f6b200711b3821d00ef3a4be8a72013268f420a0" gracePeriod=30 Sep 08 10:19:28.977698 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:28.977665 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx"] Sep 08 10:19:28.977953 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:28.977925 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" podUID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerName="registry" containerID="cri-o://d321f2eada533c356f96c7e928c6c8a3886b6a9a9fad7ea49c8d99c0cf131a40" gracePeriod=30 Sep 08 10:19:28.978109 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:28.977956 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" podUID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerName="ui" containerID="cri-o://6a9ebab4791a0c32232ff2fcf10b70be778e8b13249d09beeb98bd57c0324ec2" gracePeriod=30 Sep 08 10:19:28.978109 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:28.977971 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" podUID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerName="offline" containerID="cri-o://81375e4aae42145e0f0a7f560f7bafdf906f434b9bf3f284752c08a932e2a0cf" gracePeriod=30 Sep 08 10:19:28.978109 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:28.977974 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" podUID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerName="online" containerID="cri-o://f25a60e09810c2da1aca63de2b1498ac4f5ae993241d8ccdc16e39e391a22fdb" gracePeriod=30 Sep 08 10:19:29.385472 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:29.385441 2568 generic.go:356] "Generic (PLEG): container finished" podID="9ad9e974-8426-46fa-bb7b-32f3c6ca792c" containerID="2a8f7dfefd4bedeaafd18b2e0b0cca9dd3f6b6ab9ac6a7c426e106fbe2a9ba0e" exitCode=0 Sep 08 10:19:29.385640 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:29.385516 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" event={"ID":"9ad9e974-8426-46fa-bb7b-32f3c6ca792c","Type":"ContainerDied","Data":"2a8f7dfefd4bedeaafd18b2e0b0cca9dd3f6b6ab9ac6a7c426e106fbe2a9ba0e"} Sep 08 10:19:30.395620 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:30.395593 2568 generic.go:356] "Generic (PLEG): container finished" podID="9ad9e974-8426-46fa-bb7b-32f3c6ca792c" containerID="d3f9dc7039916686ff266ce329fec1954b5350c0461b598782b0c8919dbccb4b" exitCode=0 Sep 08 10:19:30.395996 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:30.395649 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" event={"ID":"9ad9e974-8426-46fa-bb7b-32f3c6ca792c","Type":"ContainerDied","Data":"d3f9dc7039916686ff266ce329fec1954b5350c0461b598782b0c8919dbccb4b"} Sep 08 10:19:30.398687 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:30.398662 2568 generic.go:356] "Generic (PLEG): container finished" podID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerID="6a9ebab4791a0c32232ff2fcf10b70be778e8b13249d09beeb98bd57c0324ec2" exitCode=0 Sep 08 10:19:30.398687 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:30.398684 2568 generic.go:356] "Generic (PLEG): container finished" podID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerID="f25a60e09810c2da1aca63de2b1498ac4f5ae993241d8ccdc16e39e391a22fdb" exitCode=0 Sep 08 10:19:30.402742 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:30.402719 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" event={"ID":"e256fa60-9cb7-4570-958c-63e41d145e5f","Type":"ContainerDied","Data":"6a9ebab4791a0c32232ff2fcf10b70be778e8b13249d09beeb98bd57c0324ec2"} Sep 08 10:19:30.402861 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:30.402760 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" event={"ID":"e256fa60-9cb7-4570-958c-63e41d145e5f","Type":"ContainerDied","Data":"f25a60e09810c2da1aca63de2b1498ac4f5ae993241d8ccdc16e39e391a22fdb"} Sep 08 10:19:35.268112 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:35.268070 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" podUID="9ad9e974-8426-46fa-bb7b-32f3c6ca792c" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.27:8443: connect: connection refused" Sep 08 10:19:36.133280 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:36.133239 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" podUID="9ad9e974-8426-46fa-bb7b-32f3c6ca792c" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused" Sep 08 10:19:38.597184 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:38.597136 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" podUID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused" Sep 08 10:19:38.677171 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:38.677132 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" podUID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused" Sep 08 10:19:45.268499 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:45.268459 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" podUID="9ad9e974-8426-46fa-bb7b-32f3c6ca792c" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.27:8443: connect: connection refused" Sep 08 10:19:46.133165 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:46.133131 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" podUID="9ad9e974-8426-46fa-bb7b-32f3c6ca792c" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused" Sep 08 10:19:48.596383 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:48.596331 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" podUID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused" Sep 08 10:19:48.677077 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:48.677041 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" podUID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused" Sep 08 10:19:55.268916 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:55.268865 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" podUID="9ad9e974-8426-46fa-bb7b-32f3c6ca792c" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.27:8443: connect: connection refused" Sep 08 10:19:55.269397 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:55.269023 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" Sep 08 10:19:56.132860 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:56.132824 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" podUID="9ad9e974-8426-46fa-bb7b-32f3c6ca792c" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused" Sep 08 10:19:56.133034 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:56.132941 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" Sep 08 10:19:58.597200 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:58.597157 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" podUID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused" Sep 08 10:19:58.597632 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:58.597291 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:19:58.677194 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:58.677161 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" podUID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused" Sep 08 10:19:58.677341 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:58.677271 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:19:58.944605 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:58.944575 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" Sep 08 10:19:59.057231 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.057209 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-offline-tls\") pod \"9ad9e974-8426-46fa-bb7b-32f3c6ca792c\" (UID: \"9ad9e974-8426-46fa-bb7b-32f3c6ca792c\") " Sep 08 10:19:59.057331 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.057242 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-online-tls\") pod \"9ad9e974-8426-46fa-bb7b-32f3c6ca792c\" (UID: \"9ad9e974-8426-46fa-bb7b-32f3c6ca792c\") " Sep 08 10:19:59.057331 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.057293 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-feast-data\") pod \"9ad9e974-8426-46fa-bb7b-32f3c6ca792c\" (UID: \"9ad9e974-8426-46fa-bb7b-32f3c6ca792c\") " Sep 08 10:19:59.057400 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.057344 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-ui-tls\") pod \"9ad9e974-8426-46fa-bb7b-32f3c6ca792c\" (UID: \"9ad9e974-8426-46fa-bb7b-32f3c6ca792c\") " Sep 08 10:19:59.057469 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.057400 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-registry-tls\") pod \"9ad9e974-8426-46fa-bb7b-32f3c6ca792c\" (UID: \"9ad9e974-8426-46fa-bb7b-32f3c6ca792c\") " Sep 08 10:19:59.057507 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.057478 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8hmmt\" (UniqueName: \"kubernetes.io/projected/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-kube-api-access-8hmmt\") pod \"9ad9e974-8426-46fa-bb7b-32f3c6ca792c\" (UID: \"9ad9e974-8426-46fa-bb7b-32f3c6ca792c\") " Sep 08 10:19:59.057853 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.057821 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "9ad9e974-8426-46fa-bb7b-32f3c6ca792c" (UID: "9ad9e974-8426-46fa-bb7b-32f3c6ca792c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:19:59.058055 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.057865 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-feast-data" (OuterVolumeSpecName: "feast-data") pod "9ad9e974-8426-46fa-bb7b-32f3c6ca792c" (UID: "9ad9e974-8426-46fa-bb7b-32f3c6ca792c"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 10:19:59.059569 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.059546 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "9ad9e974-8426-46fa-bb7b-32f3c6ca792c" (UID: "9ad9e974-8426-46fa-bb7b-32f3c6ca792c"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:19:59.059686 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.059665 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-kube-api-access-8hmmt" (OuterVolumeSpecName: "kube-api-access-8hmmt") pod "9ad9e974-8426-46fa-bb7b-32f3c6ca792c" (UID: "9ad9e974-8426-46fa-bb7b-32f3c6ca792c"). InnerVolumeSpecName "kube-api-access-8hmmt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 10:19:59.059853 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.059837 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-online-tls" (OuterVolumeSpecName: "online-tls") pod "9ad9e974-8426-46fa-bb7b-32f3c6ca792c" (UID: "9ad9e974-8426-46fa-bb7b-32f3c6ca792c"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:19:59.059907 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.059855 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "9ad9e974-8426-46fa-bb7b-32f3c6ca792c" (UID: "9ad9e974-8426-46fa-bb7b-32f3c6ca792c"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:19:59.158647 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.158625 2568 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-registry-tls\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:19:59.158725 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.158645 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hmmt\" (UniqueName: \"kubernetes.io/projected/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-kube-api-access-8hmmt\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:19:59.158725 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.158666 2568 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-offline-tls\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:19:59.158725 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.158674 2568 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-online-tls\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:19:59.158725 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.158681 2568 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-feast-data\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:19:59.158725 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.158688 2568 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9ad9e974-8426-46fa-bb7b-32f3c6ca792c-ui-tls\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:19:59.511927 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.511894 2568 generic.go:356] "Generic (PLEG): container finished" podID="9ad9e974-8426-46fa-bb7b-32f3c6ca792c" containerID="0586eea4019b04ade9ef2595f6b200711b3821d00ef3a4be8a72013268f420a0" exitCode=137 Sep 08 10:19:59.512045 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.511957 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" Sep 08 10:19:59.512045 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.511970 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" event={"ID":"9ad9e974-8426-46fa-bb7b-32f3c6ca792c","Type":"ContainerDied","Data":"0586eea4019b04ade9ef2595f6b200711b3821d00ef3a4be8a72013268f420a0"} Sep 08 10:19:59.512045 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.512015 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw" event={"ID":"9ad9e974-8426-46fa-bb7b-32f3c6ca792c","Type":"ContainerDied","Data":"950516065dae8e0148798770f46be78a46715d459e2564c5822aa173875f4fa4"} Sep 08 10:19:59.512045 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.512041 2568 scope.go:117] "RemoveContainer" containerID="2a8f7dfefd4bedeaafd18b2e0b0cca9dd3f6b6ab9ac6a7c426e106fbe2a9ba0e" Sep 08 10:19:59.515886 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.515862 2568 generic.go:356] "Generic (PLEG): container finished" podID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerID="81375e4aae42145e0f0a7f560f7bafdf906f434b9bf3f284752c08a932e2a0cf" exitCode=137 Sep 08 10:19:59.516009 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.515888 2568 generic.go:356] "Generic (PLEG): container finished" podID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerID="d321f2eada533c356f96c7e928c6c8a3886b6a9a9fad7ea49c8d99c0cf131a40" exitCode=137 Sep 08 10:19:59.516009 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.515934 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" event={"ID":"e256fa60-9cb7-4570-958c-63e41d145e5f","Type":"ContainerDied","Data":"81375e4aae42145e0f0a7f560f7bafdf906f434b9bf3f284752c08a932e2a0cf"} Sep 08 10:19:59.516009 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.515979 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" event={"ID":"e256fa60-9cb7-4570-958c-63e41d145e5f","Type":"ContainerDied","Data":"d321f2eada533c356f96c7e928c6c8a3886b6a9a9fad7ea49c8d99c0cf131a40"} Sep 08 10:19:59.521464 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.521442 2568 scope.go:117] "RemoveContainer" containerID="0586eea4019b04ade9ef2595f6b200711b3821d00ef3a4be8a72013268f420a0" Sep 08 10:19:59.529250 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.529231 2568 scope.go:117] "RemoveContainer" containerID="d3f9dc7039916686ff266ce329fec1954b5350c0461b598782b0c8919dbccb4b" Sep 08 10:19:59.535745 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.535725 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw"] Sep 08 10:19:59.537599 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.537565 2568 scope.go:117] "RemoveContainer" containerID="cfba0f518766fb89c67198e4e49a34bf4b1f3743d792db40503e6504831dd2fb" Sep 08 10:19:59.540220 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.540199 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-66966cfd48-w8vbw"] Sep 08 10:19:59.546779 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.546765 2568 scope.go:117] "RemoveContainer" containerID="5cf04d782734794a66f30faf6030480ca0aace1fcc44f8edfacd969b92dfeacc" Sep 08 10:19:59.555672 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.555653 2568 scope.go:117] "RemoveContainer" containerID="2a8f7dfefd4bedeaafd18b2e0b0cca9dd3f6b6ab9ac6a7c426e106fbe2a9ba0e" Sep 08 10:19:59.558465 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:19:59.558442 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a8f7dfefd4bedeaafd18b2e0b0cca9dd3f6b6ab9ac6a7c426e106fbe2a9ba0e\": container with ID starting with 2a8f7dfefd4bedeaafd18b2e0b0cca9dd3f6b6ab9ac6a7c426e106fbe2a9ba0e not found: ID does not exist" containerID="2a8f7dfefd4bedeaafd18b2e0b0cca9dd3f6b6ab9ac6a7c426e106fbe2a9ba0e" Sep 08 10:19:59.558554 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.558474 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a8f7dfefd4bedeaafd18b2e0b0cca9dd3f6b6ab9ac6a7c426e106fbe2a9ba0e"} err="failed to get container status \"2a8f7dfefd4bedeaafd18b2e0b0cca9dd3f6b6ab9ac6a7c426e106fbe2a9ba0e\": rpc error: code = NotFound desc = could not find container \"2a8f7dfefd4bedeaafd18b2e0b0cca9dd3f6b6ab9ac6a7c426e106fbe2a9ba0e\": container with ID starting with 2a8f7dfefd4bedeaafd18b2e0b0cca9dd3f6b6ab9ac6a7c426e106fbe2a9ba0e not found: ID does not exist" Sep 08 10:19:59.558554 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.558494 2568 scope.go:117] "RemoveContainer" containerID="0586eea4019b04ade9ef2595f6b200711b3821d00ef3a4be8a72013268f420a0" Sep 08 10:19:59.559005 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:19:59.558888 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0586eea4019b04ade9ef2595f6b200711b3821d00ef3a4be8a72013268f420a0\": container with ID starting with 0586eea4019b04ade9ef2595f6b200711b3821d00ef3a4be8a72013268f420a0 not found: ID does not exist" containerID="0586eea4019b04ade9ef2595f6b200711b3821d00ef3a4be8a72013268f420a0" Sep 08 10:19:59.559005 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.558918 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0586eea4019b04ade9ef2595f6b200711b3821d00ef3a4be8a72013268f420a0"} err="failed to get container status \"0586eea4019b04ade9ef2595f6b200711b3821d00ef3a4be8a72013268f420a0\": rpc error: code = NotFound desc = could not find container \"0586eea4019b04ade9ef2595f6b200711b3821d00ef3a4be8a72013268f420a0\": container with ID starting with 0586eea4019b04ade9ef2595f6b200711b3821d00ef3a4be8a72013268f420a0 not found: ID does not exist" Sep 08 10:19:59.559005 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.558940 2568 scope.go:117] "RemoveContainer" containerID="d3f9dc7039916686ff266ce329fec1954b5350c0461b598782b0c8919dbccb4b" Sep 08 10:19:59.559247 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:19:59.559221 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3f9dc7039916686ff266ce329fec1954b5350c0461b598782b0c8919dbccb4b\": container with ID starting with d3f9dc7039916686ff266ce329fec1954b5350c0461b598782b0c8919dbccb4b not found: ID does not exist" containerID="d3f9dc7039916686ff266ce329fec1954b5350c0461b598782b0c8919dbccb4b" Sep 08 10:19:59.559303 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.559258 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3f9dc7039916686ff266ce329fec1954b5350c0461b598782b0c8919dbccb4b"} err="failed to get container status \"d3f9dc7039916686ff266ce329fec1954b5350c0461b598782b0c8919dbccb4b\": rpc error: code = NotFound desc = could not find container \"d3f9dc7039916686ff266ce329fec1954b5350c0461b598782b0c8919dbccb4b\": container with ID starting with d3f9dc7039916686ff266ce329fec1954b5350c0461b598782b0c8919dbccb4b not found: ID does not exist" Sep 08 10:19:59.559303 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.559279 2568 scope.go:117] "RemoveContainer" containerID="cfba0f518766fb89c67198e4e49a34bf4b1f3743d792db40503e6504831dd2fb" Sep 08 10:19:59.559642 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:19:59.559609 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfba0f518766fb89c67198e4e49a34bf4b1f3743d792db40503e6504831dd2fb\": container with ID starting with cfba0f518766fb89c67198e4e49a34bf4b1f3743d792db40503e6504831dd2fb not found: ID does not exist" containerID="cfba0f518766fb89c67198e4e49a34bf4b1f3743d792db40503e6504831dd2fb" Sep 08 10:19:59.559746 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.559650 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfba0f518766fb89c67198e4e49a34bf4b1f3743d792db40503e6504831dd2fb"} err="failed to get container status \"cfba0f518766fb89c67198e4e49a34bf4b1f3743d792db40503e6504831dd2fb\": rpc error: code = NotFound desc = could not find container \"cfba0f518766fb89c67198e4e49a34bf4b1f3743d792db40503e6504831dd2fb\": container with ID starting with cfba0f518766fb89c67198e4e49a34bf4b1f3743d792db40503e6504831dd2fb not found: ID does not exist" Sep 08 10:19:59.559746 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.559669 2568 scope.go:117] "RemoveContainer" containerID="5cf04d782734794a66f30faf6030480ca0aace1fcc44f8edfacd969b92dfeacc" Sep 08 10:19:59.560054 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:19:59.560031 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cf04d782734794a66f30faf6030480ca0aace1fcc44f8edfacd969b92dfeacc\": container with ID starting with 5cf04d782734794a66f30faf6030480ca0aace1fcc44f8edfacd969b92dfeacc not found: ID does not exist" containerID="5cf04d782734794a66f30faf6030480ca0aace1fcc44f8edfacd969b92dfeacc" Sep 08 10:19:59.560134 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.560060 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cf04d782734794a66f30faf6030480ca0aace1fcc44f8edfacd969b92dfeacc"} err="failed to get container status \"5cf04d782734794a66f30faf6030480ca0aace1fcc44f8edfacd969b92dfeacc\": rpc error: code = NotFound desc = could not find container \"5cf04d782734794a66f30faf6030480ca0aace1fcc44f8edfacd969b92dfeacc\": container with ID starting with 5cf04d782734794a66f30faf6030480ca0aace1fcc44f8edfacd969b92dfeacc not found: ID does not exist" Sep 08 10:19:59.617641 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.617618 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:19:59.762472 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.762377 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e256fa60-9cb7-4570-958c-63e41d145e5f-ui-tls\") pod \"e256fa60-9cb7-4570-958c-63e41d145e5f\" (UID: \"e256fa60-9cb7-4570-958c-63e41d145e5f\") " Sep 08 10:19:59.762472 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.762424 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e256fa60-9cb7-4570-958c-63e41d145e5f-offline-tls\") pod \"e256fa60-9cb7-4570-958c-63e41d145e5f\" (UID: \"e256fa60-9cb7-4570-958c-63e41d145e5f\") " Sep 08 10:19:59.762472 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.762443 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e256fa60-9cb7-4570-958c-63e41d145e5f-registry-tls\") pod \"e256fa60-9cb7-4570-958c-63e41d145e5f\" (UID: \"e256fa60-9cb7-4570-958c-63e41d145e5f\") " Sep 08 10:19:59.762772 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.762495 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j5v4t\" (UniqueName: \"kubernetes.io/projected/e256fa60-9cb7-4570-958c-63e41d145e5f-kube-api-access-j5v4t\") pod \"e256fa60-9cb7-4570-958c-63e41d145e5f\" (UID: \"e256fa60-9cb7-4570-958c-63e41d145e5f\") " Sep 08 10:19:59.762772 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.762516 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e256fa60-9cb7-4570-958c-63e41d145e5f-online-tls\") pod \"e256fa60-9cb7-4570-958c-63e41d145e5f\" (UID: \"e256fa60-9cb7-4570-958c-63e41d145e5f\") " Sep 08 10:19:59.762772 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.762552 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e256fa60-9cb7-4570-958c-63e41d145e5f-feast-data\") pod \"e256fa60-9cb7-4570-958c-63e41d145e5f\" (UID: \"e256fa60-9cb7-4570-958c-63e41d145e5f\") " Sep 08 10:19:59.765573 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.765537 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e256fa60-9cb7-4570-958c-63e41d145e5f-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "e256fa60-9cb7-4570-958c-63e41d145e5f" (UID: "e256fa60-9cb7-4570-958c-63e41d145e5f"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:19:59.765777 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.765751 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e256fa60-9cb7-4570-958c-63e41d145e5f-feast-data" (OuterVolumeSpecName: "feast-data") pod "e256fa60-9cb7-4570-958c-63e41d145e5f" (UID: "e256fa60-9cb7-4570-958c-63e41d145e5f"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 10:19:59.770226 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.770191 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e256fa60-9cb7-4570-958c-63e41d145e5f-kube-api-access-j5v4t" (OuterVolumeSpecName: "kube-api-access-j5v4t") pod "e256fa60-9cb7-4570-958c-63e41d145e5f" (UID: "e256fa60-9cb7-4570-958c-63e41d145e5f"). InnerVolumeSpecName "kube-api-access-j5v4t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 10:19:59.770595 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.770498 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e256fa60-9cb7-4570-958c-63e41d145e5f-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "e256fa60-9cb7-4570-958c-63e41d145e5f" (UID: "e256fa60-9cb7-4570-958c-63e41d145e5f"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:19:59.770595 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.770529 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e256fa60-9cb7-4570-958c-63e41d145e5f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e256fa60-9cb7-4570-958c-63e41d145e5f" (UID: "e256fa60-9cb7-4570-958c-63e41d145e5f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:19:59.771367 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.771346 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e256fa60-9cb7-4570-958c-63e41d145e5f-online-tls" (OuterVolumeSpecName: "online-tls") pod "e256fa60-9cb7-4570-958c-63e41d145e5f" (UID: "e256fa60-9cb7-4570-958c-63e41d145e5f"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:19:59.864018 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.863992 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5v4t\" (UniqueName: \"kubernetes.io/projected/e256fa60-9cb7-4570-958c-63e41d145e5f-kube-api-access-j5v4t\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:19:59.864018 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.864015 2568 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e256fa60-9cb7-4570-958c-63e41d145e5f-online-tls\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:19:59.864170 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.864029 2568 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e256fa60-9cb7-4570-958c-63e41d145e5f-feast-data\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:19:59.864170 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.864040 2568 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e256fa60-9cb7-4570-958c-63e41d145e5f-ui-tls\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:19:59.864170 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.864050 2568 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e256fa60-9cb7-4570-958c-63e41d145e5f-offline-tls\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:19:59.864170 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:19:59.864060 2568 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e256fa60-9cb7-4570-958c-63e41d145e5f-registry-tls\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:20:00.403802 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:00.403772 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ad9e974-8426-46fa-bb7b-32f3c6ca792c" path="/var/lib/kubelet/pods/9ad9e974-8426-46fa-bb7b-32f3c6ca792c/volumes" Sep 08 10:20:00.523262 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:00.523236 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" event={"ID":"e256fa60-9cb7-4570-958c-63e41d145e5f","Type":"ContainerDied","Data":"c4cfe8880414f3a55b920f973950a6908a350103d12050d01b8adb0ebbb7497a"} Sep 08 10:20:00.523377 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:00.523287 2568 scope.go:117] "RemoveContainer" containerID="6a9ebab4791a0c32232ff2fcf10b70be778e8b13249d09beeb98bd57c0324ec2" Sep 08 10:20:00.523377 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:00.523309 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx" Sep 08 10:20:00.531203 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:00.531186 2568 scope.go:117] "RemoveContainer" containerID="81375e4aae42145e0f0a7f560f7bafdf906f434b9bf3f284752c08a932e2a0cf" Sep 08 10:20:00.538900 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:00.538882 2568 scope.go:117] "RemoveContainer" containerID="f25a60e09810c2da1aca63de2b1498ac4f5ae993241d8ccdc16e39e391a22fdb" Sep 08 10:20:00.545980 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:00.545966 2568 scope.go:117] "RemoveContainer" containerID="d321f2eada533c356f96c7e928c6c8a3886b6a9a9fad7ea49c8d99c0cf131a40" Sep 08 10:20:00.553712 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:00.553692 2568 scope.go:117] "RemoveContainer" containerID="891b09eb803c6dce6022d83b53c65cd535ed65ed8588bfdcbb66ae65a7cdb568" Sep 08 10:20:00.554147 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:00.554129 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx"] Sep 08 10:20:00.555555 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:00.555511 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-b5b55ccfc-ww8zx"] Sep 08 10:20:00.560857 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:00.560837 2568 scope.go:117] "RemoveContainer" containerID="9d15458c9589b665025bff2cd24631b454b8613e0da957c273188dc21f272eef" Sep 08 10:20:02.404708 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:02.404675 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e256fa60-9cb7-4570-958c-63e41d145e5f" path="/var/lib/kubelet/pods/e256fa60-9cb7-4570-958c-63e41d145e5f/volumes" Sep 08 10:20:12.315005 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.314972 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/postgres-864846d4f5-8pgcm"] Sep 08 10:20:12.315367 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.315317 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerName="online" Sep 08 10:20:12.315367 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.315331 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerName="online" Sep 08 10:20:12.315367 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.315340 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerName="offline" Sep 08 10:20:12.315367 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.315345 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerName="offline" Sep 08 10:20:12.315367 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.315352 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerName="ui" Sep 08 10:20:12.315367 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.315357 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerName="ui" Sep 08 10:20:12.315367 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.315368 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ad9e974-8426-46fa-bb7b-32f3c6ca792c" containerName="online" Sep 08 10:20:12.315605 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.315372 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ad9e974-8426-46fa-bb7b-32f3c6ca792c" containerName="online" Sep 08 10:20:12.315605 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.315382 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ad9e974-8426-46fa-bb7b-32f3c6ca792c" containerName="ui" Sep 08 10:20:12.315605 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.315389 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ad9e974-8426-46fa-bb7b-32f3c6ca792c" containerName="ui" Sep 08 10:20:12.315605 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.315397 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerName="feast-apply" Sep 08 10:20:12.315605 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.315402 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerName="feast-apply" Sep 08 10:20:12.315605 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.315421 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ad9e974-8426-46fa-bb7b-32f3c6ca792c" containerName="offline" Sep 08 10:20:12.315605 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.315426 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ad9e974-8426-46fa-bb7b-32f3c6ca792c" containerName="offline" Sep 08 10:20:12.315605 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.315433 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ad9e974-8426-46fa-bb7b-32f3c6ca792c" containerName="feast-apply" Sep 08 10:20:12.315605 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.315438 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ad9e974-8426-46fa-bb7b-32f3c6ca792c" containerName="feast-apply" Sep 08 10:20:12.315605 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.315444 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerName="feast-init" Sep 08 10:20:12.315605 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.315449 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerName="feast-init" Sep 08 10:20:12.315605 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.315457 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerName="registry" Sep 08 10:20:12.315605 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.315462 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerName="registry" Sep 08 10:20:12.315605 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.315467 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ad9e974-8426-46fa-bb7b-32f3c6ca792c" containerName="feast-init" Sep 08 10:20:12.315605 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.315472 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ad9e974-8426-46fa-bb7b-32f3c6ca792c" containerName="feast-init" Sep 08 10:20:12.315605 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.315517 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ad9e974-8426-46fa-bb7b-32f3c6ca792c" containerName="offline" Sep 08 10:20:12.315605 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.315527 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerName="offline" Sep 08 10:20:12.315605 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.315532 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ad9e974-8426-46fa-bb7b-32f3c6ca792c" containerName="online" Sep 08 10:20:12.315605 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.315539 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ad9e974-8426-46fa-bb7b-32f3c6ca792c" containerName="ui" Sep 08 10:20:12.315605 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.315544 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerName="registry" Sep 08 10:20:12.315605 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.315550 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerName="online" Sep 08 10:20:12.315605 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.315554 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="e256fa60-9cb7-4570-958c-63e41d145e5f" containerName="ui" Sep 08 10:20:12.320251 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.320229 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-8pgcm" Sep 08 10:20:12.323672 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.323649 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\"" Sep 08 10:20:12.323803 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.323773 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 08 10:20:12.324782 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.324757 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-qlgmz\"" Sep 08 10:20:12.324782 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.324776 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 08 10:20:12.326523 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.326497 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-8pgcm"] Sep 08 10:20:12.451624 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.451591 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/a5d3466f-c146-4965-9f26-880204265c7b-postgresdata\") pod \"postgres-864846d4f5-8pgcm\" (UID: \"a5d3466f-c146-4965-9f26-880204265c7b\") " pod="test-ns-feast/postgres-864846d4f5-8pgcm" Sep 08 10:20:12.451786 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.451639 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2glr2\" (UniqueName: \"kubernetes.io/projected/a5d3466f-c146-4965-9f26-880204265c7b-kube-api-access-2glr2\") pod \"postgres-864846d4f5-8pgcm\" (UID: \"a5d3466f-c146-4965-9f26-880204265c7b\") " pod="test-ns-feast/postgres-864846d4f5-8pgcm" Sep 08 10:20:12.552190 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.552161 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2glr2\" (UniqueName: \"kubernetes.io/projected/a5d3466f-c146-4965-9f26-880204265c7b-kube-api-access-2glr2\") pod \"postgres-864846d4f5-8pgcm\" (UID: \"a5d3466f-c146-4965-9f26-880204265c7b\") " pod="test-ns-feast/postgres-864846d4f5-8pgcm" Sep 08 10:20:12.552364 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.552248 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/a5d3466f-c146-4965-9f26-880204265c7b-postgresdata\") pod \"postgres-864846d4f5-8pgcm\" (UID: \"a5d3466f-c146-4965-9f26-880204265c7b\") " pod="test-ns-feast/postgres-864846d4f5-8pgcm" Sep 08 10:20:12.552629 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.552608 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/a5d3466f-c146-4965-9f26-880204265c7b-postgresdata\") pod \"postgres-864846d4f5-8pgcm\" (UID: \"a5d3466f-c146-4965-9f26-880204265c7b\") " pod="test-ns-feast/postgres-864846d4f5-8pgcm" Sep 08 10:20:12.561129 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.561107 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2glr2\" (UniqueName: \"kubernetes.io/projected/a5d3466f-c146-4965-9f26-880204265c7b-kube-api-access-2glr2\") pod \"postgres-864846d4f5-8pgcm\" (UID: \"a5d3466f-c146-4965-9f26-880204265c7b\") " pod="test-ns-feast/postgres-864846d4f5-8pgcm" Sep 08 10:20:12.648492 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.648467 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-8pgcm" Sep 08 10:20:12.917719 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.917639 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-8pgcm"] Sep 08 10:20:12.920774 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:20:12.920750 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda5d3466f_c146_4965_9f26_880204265c7b.slice/crio-d018b70ad92f4fc7e173719db6e1e3b098bda077cdc0e6630dd06074e950ca68 WatchSource:0}: Error finding container d018b70ad92f4fc7e173719db6e1e3b098bda077cdc0e6630dd06074e950ca68: Status 404 returned error can't find the container with id d018b70ad92f4fc7e173719db6e1e3b098bda077cdc0e6630dd06074e950ca68 Sep 08 10:20:12.922618 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:12.922603 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 08 10:20:13.575133 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:13.575092 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-8pgcm" event={"ID":"a5d3466f-c146-4965-9f26-880204265c7b","Type":"ContainerStarted","Data":"d018b70ad92f4fc7e173719db6e1e3b098bda077cdc0e6630dd06074e950ca68"} Sep 08 10:20:18.599868 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:18.599832 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-8pgcm" event={"ID":"a5d3466f-c146-4965-9f26-880204265c7b","Type":"ContainerStarted","Data":"2557252b0a511fcdb81d9f99fdd12aee9484ec2a8649df72d78df48e47ca95d7"} Sep 08 10:20:18.623188 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:18.623129 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/postgres-864846d4f5-8pgcm" podStartSLOduration=1.489200499 podStartE2EDuration="6.623115662s" totalImagesPullingTime="5.133915163s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:20:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:20:18.62299419 +0000 UTC m=+802.667098256" watchObservedRunningTime="2026-09-08 10:20:18.623115662 +0000 UTC m=+802.667219730" Sep 08 10:20:23.244246 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:23.244209 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl"] Sep 08 10:20:23.247567 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:23.247550 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" Sep 08 10:20:23.251179 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:23.251155 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-registry-tls\"" Sep 08 10:20:23.251317 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:23.251203 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-8p8ps\"" Sep 08 10:20:23.251317 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:23.251225 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Sep 08 10:20:23.251529 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:23.251501 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-online-tls\"" Sep 08 10:20:23.251828 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:23.251797 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-offline-tls\"" Sep 08 10:20:23.266985 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:23.266962 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl"] Sep 08 10:20:23.438881 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:23.438852 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns56p\" (UniqueName: \"kubernetes.io/projected/b660b54b-5c60-483c-a01f-04a20247bc7f-kube-api-access-ns56p\") pod \"feast-credit-scoring-76868fd94f-zrbvl\" (UID: \"b660b54b-5c60-483c-a01f-04a20247bc7f\") " pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" Sep 08 10:20:23.439026 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:23.438889 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b660b54b-5c60-483c-a01f-04a20247bc7f-offline-tls\") pod \"feast-credit-scoring-76868fd94f-zrbvl\" (UID: \"b660b54b-5c60-483c-a01f-04a20247bc7f\") " pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" Sep 08 10:20:23.439026 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:23.438922 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b660b54b-5c60-483c-a01f-04a20247bc7f-registry-tls\") pod \"feast-credit-scoring-76868fd94f-zrbvl\" (UID: \"b660b54b-5c60-483c-a01f-04a20247bc7f\") " pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" Sep 08 10:20:23.439026 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:23.438940 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b660b54b-5c60-483c-a01f-04a20247bc7f-online-tls\") pod \"feast-credit-scoring-76868fd94f-zrbvl\" (UID: \"b660b54b-5c60-483c-a01f-04a20247bc7f\") " pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" Sep 08 10:20:23.439150 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:23.439063 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b660b54b-5c60-483c-a01f-04a20247bc7f-feast-data\") pod \"feast-credit-scoring-76868fd94f-zrbvl\" (UID: \"b660b54b-5c60-483c-a01f-04a20247bc7f\") " pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" Sep 08 10:20:23.539520 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:23.539441 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b660b54b-5c60-483c-a01f-04a20247bc7f-offline-tls\") pod \"feast-credit-scoring-76868fd94f-zrbvl\" (UID: \"b660b54b-5c60-483c-a01f-04a20247bc7f\") " pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" Sep 08 10:20:23.539520 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:23.539485 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b660b54b-5c60-483c-a01f-04a20247bc7f-registry-tls\") pod \"feast-credit-scoring-76868fd94f-zrbvl\" (UID: \"b660b54b-5c60-483c-a01f-04a20247bc7f\") " pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" Sep 08 10:20:23.539520 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:23.539505 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b660b54b-5c60-483c-a01f-04a20247bc7f-online-tls\") pod \"feast-credit-scoring-76868fd94f-zrbvl\" (UID: \"b660b54b-5c60-483c-a01f-04a20247bc7f\") " pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" Sep 08 10:20:23.539743 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:23.539551 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b660b54b-5c60-483c-a01f-04a20247bc7f-feast-data\") pod \"feast-credit-scoring-76868fd94f-zrbvl\" (UID: \"b660b54b-5c60-483c-a01f-04a20247bc7f\") " pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" Sep 08 10:20:23.539743 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:23.539573 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns56p\" (UniqueName: \"kubernetes.io/projected/b660b54b-5c60-483c-a01f-04a20247bc7f-kube-api-access-ns56p\") pod \"feast-credit-scoring-76868fd94f-zrbvl\" (UID: \"b660b54b-5c60-483c-a01f-04a20247bc7f\") " pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" Sep 08 10:20:23.540057 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:23.540038 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b660b54b-5c60-483c-a01f-04a20247bc7f-feast-data\") pod \"feast-credit-scoring-76868fd94f-zrbvl\" (UID: \"b660b54b-5c60-483c-a01f-04a20247bc7f\") " pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" Sep 08 10:20:23.542173 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:23.542143 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b660b54b-5c60-483c-a01f-04a20247bc7f-offline-tls\") pod \"feast-credit-scoring-76868fd94f-zrbvl\" (UID: \"b660b54b-5c60-483c-a01f-04a20247bc7f\") " pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" Sep 08 10:20:23.542277 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:23.542175 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b660b54b-5c60-483c-a01f-04a20247bc7f-registry-tls\") pod \"feast-credit-scoring-76868fd94f-zrbvl\" (UID: \"b660b54b-5c60-483c-a01f-04a20247bc7f\") " pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" Sep 08 10:20:23.542277 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:23.542236 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b660b54b-5c60-483c-a01f-04a20247bc7f-online-tls\") pod \"feast-credit-scoring-76868fd94f-zrbvl\" (UID: \"b660b54b-5c60-483c-a01f-04a20247bc7f\") " pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" Sep 08 10:20:23.547807 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:23.547789 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns56p\" (UniqueName: \"kubernetes.io/projected/b660b54b-5c60-483c-a01f-04a20247bc7f-kube-api-access-ns56p\") pod \"feast-credit-scoring-76868fd94f-zrbvl\" (UID: \"b660b54b-5c60-483c-a01f-04a20247bc7f\") " pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" Sep 08 10:20:23.558699 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:23.558681 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" Sep 08 10:20:23.833811 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:23.833789 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl"] Sep 08 10:20:23.835936 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:20:23.835900 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb660b54b_5c60_483c_a01f_04a20247bc7f.slice/crio-05b176e69a601f2f14619b411bb83a01bd3db6239e76c399075f87aa3437b6ff WatchSource:0}: Error finding container 05b176e69a601f2f14619b411bb83a01bd3db6239e76c399075f87aa3437b6ff: Status 404 returned error can't find the container with id 05b176e69a601f2f14619b411bb83a01bd3db6239e76c399075f87aa3437b6ff Sep 08 10:20:24.626215 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:24.626180 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" event={"ID":"b660b54b-5c60-483c-a01f-04a20247bc7f","Type":"ContainerStarted","Data":"2957c34ffcbcb1a578b5d33cf497a4c0a718d7de826519b68ead54807cb05079"} Sep 08 10:20:24.626594 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:24.626239 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" event={"ID":"b660b54b-5c60-483c-a01f-04a20247bc7f","Type":"ContainerStarted","Data":"05b176e69a601f2f14619b411bb83a01bd3db6239e76c399075f87aa3437b6ff"} Sep 08 10:20:25.631653 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:25.631625 2568 generic.go:356] "Generic (PLEG): container finished" podID="b660b54b-5c60-483c-a01f-04a20247bc7f" containerID="2957c34ffcbcb1a578b5d33cf497a4c0a718d7de826519b68ead54807cb05079" exitCode=0 Sep 08 10:20:25.632078 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:25.631715 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" event={"ID":"b660b54b-5c60-483c-a01f-04a20247bc7f","Type":"ContainerDied","Data":"2957c34ffcbcb1a578b5d33cf497a4c0a718d7de826519b68ead54807cb05079"} Sep 08 10:20:26.637467 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:26.637396 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" event={"ID":"b660b54b-5c60-483c-a01f-04a20247bc7f","Type":"ContainerStarted","Data":"ec27010a1662154ecf211a9b7ae8523359f0d85614224aaaddc1c22a8e03d6a6"} Sep 08 10:20:32.662841 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:32.662810 2568 generic.go:356] "Generic (PLEG): container finished" podID="b660b54b-5c60-483c-a01f-04a20247bc7f" containerID="ec27010a1662154ecf211a9b7ae8523359f0d85614224aaaddc1c22a8e03d6a6" exitCode=0 Sep 08 10:20:32.663248 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:32.662850 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" event={"ID":"b660b54b-5c60-483c-a01f-04a20247bc7f","Type":"ContainerDied","Data":"ec27010a1662154ecf211a9b7ae8523359f0d85614224aaaddc1c22a8e03d6a6"} Sep 08 10:20:33.671169 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:33.671132 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" event={"ID":"b660b54b-5c60-483c-a01f-04a20247bc7f","Type":"ContainerStarted","Data":"f338919f7da33673e884c978a26b7b76651fec789cc5ef3fb4acd7b43a586678"} Sep 08 10:20:33.671563 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:33.671214 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" event={"ID":"b660b54b-5c60-483c-a01f-04a20247bc7f","Type":"ContainerStarted","Data":"4021fbf21a9e02dbb48994bbb0c933c756de988b2bd7cf7cd1e74fbf62482192"} Sep 08 10:20:33.671563 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:33.671244 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" event={"ID":"b660b54b-5c60-483c-a01f-04a20247bc7f","Type":"ContainerStarted","Data":"233b0a812abf389225575b44d461ba13786b3f95dd6677c6f39b486f85882b72"} Sep 08 10:20:33.697675 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:33.697625 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" podStartSLOduration=10.697606899 podStartE2EDuration="10.697606899s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:20:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:20:33.695710071 +0000 UTC m=+817.739814183" watchObservedRunningTime="2026-09-08 10:20:33.697606899 +0000 UTC m=+817.741710967" Sep 08 10:20:35.559207 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:35.559170 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" Sep 08 10:20:35.559822 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:35.559791 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" Sep 08 10:20:35.559948 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:35.559833 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" Sep 08 10:20:35.560937 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:35.560906 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" podUID="b660b54b-5c60-483c-a01f-04a20247bc7f" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.29:8816: connect: connection refused" Sep 08 10:20:35.561033 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:35.560943 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" podUID="b660b54b-5c60-483c-a01f-04a20247bc7f" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.29:6571: connect: connection refused" Sep 08 10:20:35.561033 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:35.560944 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" podUID="b660b54b-5c60-483c-a01f-04a20247bc7f" containerName="online" probeResult="failure" output="Get \"https://10.134.0.29:6567/health\": dial tcp 10.134.0.29:6567: connect: connection refused" Sep 08 10:20:38.559143 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:38.559098 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" podUID="b660b54b-5c60-483c-a01f-04a20247bc7f" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.29:8816: connect: connection refused" Sep 08 10:20:38.559773 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:38.559093 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" podUID="b660b54b-5c60-483c-a01f-04a20247bc7f" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.29:6571: connect: connection refused" Sep 08 10:20:38.559907 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:38.559141 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" podUID="b660b54b-5c60-483c-a01f-04a20247bc7f" containerName="online" probeResult="failure" output="Get \"https://10.134.0.29:6567/health\": dial tcp 10.134.0.29:6567: connect: connection refused" Sep 08 10:20:41.559519 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:41.559475 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" podUID="b660b54b-5c60-483c-a01f-04a20247bc7f" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.29:6571: connect: connection refused" Sep 08 10:20:41.560124 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:41.560105 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" Sep 08 10:20:41.560560 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:41.560533 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" Sep 08 10:20:41.561010 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:41.560989 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" Sep 08 10:20:41.565837 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:41.565816 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" Sep 08 10:20:41.716024 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:41.715997 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" Sep 08 10:20:41.719338 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:41.719316 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" Sep 08 10:20:43.559011 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:43.558976 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" Sep 08 10:20:44.559775 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:44.559727 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" podUID="b660b54b-5c60-483c-a01f-04a20247bc7f" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.29:6571: connect: connection refused" Sep 08 10:20:47.560480 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:47.560450 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" Sep 08 10:20:47.561240 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:20:47.561220 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" Sep 08 10:21:52.004341 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:21:52.004265 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl"] Sep 08 10:21:52.004894 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:21:52.004522 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" podUID="b660b54b-5c60-483c-a01f-04a20247bc7f" containerName="registry" containerID="cri-o://233b0a812abf389225575b44d461ba13786b3f95dd6677c6f39b486f85882b72" gracePeriod=30 Sep 08 10:21:52.004894 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:21:52.004629 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" podUID="b660b54b-5c60-483c-a01f-04a20247bc7f" containerName="offline" containerID="cri-o://f338919f7da33673e884c978a26b7b76651fec789cc5ef3fb4acd7b43a586678" gracePeriod=30 Sep 08 10:21:52.004894 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:21:52.004635 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" podUID="b660b54b-5c60-483c-a01f-04a20247bc7f" containerName="online" containerID="cri-o://4021fbf21a9e02dbb48994bbb0c933c756de988b2bd7cf7cd1e74fbf62482192" gracePeriod=30 Sep 08 10:21:52.024681 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:21:52.024656 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-8pgcm"] Sep 08 10:21:52.024821 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:21:52.024804 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/postgres-864846d4f5-8pgcm" podUID="a5d3466f-c146-4965-9f26-880204265c7b" containerName="postgres" containerID="cri-o://2557252b0a511fcdb81d9f99fdd12aee9484ec2a8649df72d78df48e47ca95d7" gracePeriod=30 Sep 08 10:21:52.268151 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:21:52.268123 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-8pgcm" Sep 08 10:21:52.310218 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:21:52.310191 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2glr2\" (UniqueName: \"kubernetes.io/projected/a5d3466f-c146-4965-9f26-880204265c7b-kube-api-access-2glr2\") pod \"a5d3466f-c146-4965-9f26-880204265c7b\" (UID: \"a5d3466f-c146-4965-9f26-880204265c7b\") " Sep 08 10:21:52.310377 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:21:52.310262 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/a5d3466f-c146-4965-9f26-880204265c7b-postgresdata\") pod \"a5d3466f-c146-4965-9f26-880204265c7b\" (UID: \"a5d3466f-c146-4965-9f26-880204265c7b\") " Sep 08 10:21:52.312521 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:21:52.312486 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5d3466f-c146-4965-9f26-880204265c7b-kube-api-access-2glr2" (OuterVolumeSpecName: "kube-api-access-2glr2") pod "a5d3466f-c146-4965-9f26-880204265c7b" (UID: "a5d3466f-c146-4965-9f26-880204265c7b"). InnerVolumeSpecName "kube-api-access-2glr2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 10:21:52.341958 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:21:52.341928 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5d3466f-c146-4965-9f26-880204265c7b-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "a5d3466f-c146-4965-9f26-880204265c7b" (UID: "a5d3466f-c146-4965-9f26-880204265c7b"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 10:21:52.410988 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:21:52.410954 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2glr2\" (UniqueName: \"kubernetes.io/projected/a5d3466f-c146-4965-9f26-880204265c7b-kube-api-access-2glr2\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:21:52.410988 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:21:52.410989 2568 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/a5d3466f-c146-4965-9f26-880204265c7b-postgresdata\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:21:53.003591 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:21:53.003559 2568 generic.go:356] "Generic (PLEG): container finished" podID="b660b54b-5c60-483c-a01f-04a20247bc7f" containerID="4021fbf21a9e02dbb48994bbb0c933c756de988b2bd7cf7cd1e74fbf62482192" exitCode=0 Sep 08 10:21:53.003722 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:21:53.003631 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" event={"ID":"b660b54b-5c60-483c-a01f-04a20247bc7f","Type":"ContainerDied","Data":"4021fbf21a9e02dbb48994bbb0c933c756de988b2bd7cf7cd1e74fbf62482192"} Sep 08 10:21:53.004812 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:21:53.004794 2568 generic.go:356] "Generic (PLEG): container finished" podID="a5d3466f-c146-4965-9f26-880204265c7b" containerID="2557252b0a511fcdb81d9f99fdd12aee9484ec2a8649df72d78df48e47ca95d7" exitCode=0 Sep 08 10:21:53.005138 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:21:53.004836 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-8pgcm" event={"ID":"a5d3466f-c146-4965-9f26-880204265c7b","Type":"ContainerDied","Data":"2557252b0a511fcdb81d9f99fdd12aee9484ec2a8649df72d78df48e47ca95d7"} Sep 08 10:21:53.005138 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:21:53.004854 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-8pgcm" Sep 08 10:21:53.005138 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:21:53.004865 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-8pgcm" event={"ID":"a5d3466f-c146-4965-9f26-880204265c7b","Type":"ContainerDied","Data":"d018b70ad92f4fc7e173719db6e1e3b098bda077cdc0e6630dd06074e950ca68"} Sep 08 10:21:53.005138 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:21:53.004887 2568 scope.go:117] "RemoveContainer" containerID="2557252b0a511fcdb81d9f99fdd12aee9484ec2a8649df72d78df48e47ca95d7" Sep 08 10:21:53.012502 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:21:53.012483 2568 scope.go:117] "RemoveContainer" containerID="2557252b0a511fcdb81d9f99fdd12aee9484ec2a8649df72d78df48e47ca95d7" Sep 08 10:21:53.012782 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:21:53.012766 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2557252b0a511fcdb81d9f99fdd12aee9484ec2a8649df72d78df48e47ca95d7\": container with ID starting with 2557252b0a511fcdb81d9f99fdd12aee9484ec2a8649df72d78df48e47ca95d7 not found: ID does not exist" containerID="2557252b0a511fcdb81d9f99fdd12aee9484ec2a8649df72d78df48e47ca95d7" Sep 08 10:21:53.012832 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:21:53.012790 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2557252b0a511fcdb81d9f99fdd12aee9484ec2a8649df72d78df48e47ca95d7"} err="failed to get container status \"2557252b0a511fcdb81d9f99fdd12aee9484ec2a8649df72d78df48e47ca95d7\": rpc error: code = NotFound desc = could not find container \"2557252b0a511fcdb81d9f99fdd12aee9484ec2a8649df72d78df48e47ca95d7\": container with ID starting with 2557252b0a511fcdb81d9f99fdd12aee9484ec2a8649df72d78df48e47ca95d7 not found: ID does not exist" Sep 08 10:21:53.023371 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:21:53.023350 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-8pgcm"] Sep 08 10:21:53.029835 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:21:53.029817 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/postgres-864846d4f5-8pgcm"] Sep 08 10:21:54.403667 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:21:54.403636 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5d3466f-c146-4965-9f26-880204265c7b" path="/var/lib/kubelet/pods/a5d3466f-c146-4965-9f26-880204265c7b/volumes" Sep 08 10:22:01.716737 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:22:01.716699 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" podUID="b660b54b-5c60-483c-a01f-04a20247bc7f" containerName="online" probeResult="failure" output="Get \"https://10.134.0.29:6567/health\": dial tcp 10.134.0.29:6567: connect: connection refused" Sep 08 10:22:11.716079 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:22:11.716029 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" podUID="b660b54b-5c60-483c-a01f-04a20247bc7f" containerName="online" probeResult="failure" output="Get \"https://10.134.0.29:6567/health\": dial tcp 10.134.0.29:6567: connect: connection refused" Sep 08 10:22:21.716481 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:22:21.716442 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" podUID="b660b54b-5c60-483c-a01f-04a20247bc7f" containerName="online" probeResult="failure" output="Get \"https://10.134.0.29:6567/health\": dial tcp 10.134.0.29:6567: connect: connection refused" Sep 08 10:22:21.716824 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:22:21.716545 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" Sep 08 10:22:22.120703 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:22:22.120679 2568 generic.go:356] "Generic (PLEG): container finished" podID="b660b54b-5c60-483c-a01f-04a20247bc7f" containerID="f338919f7da33673e884c978a26b7b76651fec789cc5ef3fb4acd7b43a586678" exitCode=137 Sep 08 10:22:22.120703 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:22:22.120701 2568 generic.go:356] "Generic (PLEG): container finished" podID="b660b54b-5c60-483c-a01f-04a20247bc7f" containerID="233b0a812abf389225575b44d461ba13786b3f95dd6677c6f39b486f85882b72" exitCode=137 Sep 08 10:22:22.120858 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:22:22.120757 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" event={"ID":"b660b54b-5c60-483c-a01f-04a20247bc7f","Type":"ContainerDied","Data":"f338919f7da33673e884c978a26b7b76651fec789cc5ef3fb4acd7b43a586678"} Sep 08 10:22:22.120858 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:22:22.120810 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" event={"ID":"b660b54b-5c60-483c-a01f-04a20247bc7f","Type":"ContainerDied","Data":"233b0a812abf389225575b44d461ba13786b3f95dd6677c6f39b486f85882b72"} Sep 08 10:22:22.649035 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:22:22.649012 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" Sep 08 10:22:22.715352 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:22:22.715328 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b660b54b-5c60-483c-a01f-04a20247bc7f-feast-data\") pod \"b660b54b-5c60-483c-a01f-04a20247bc7f\" (UID: \"b660b54b-5c60-483c-a01f-04a20247bc7f\") " Sep 08 10:22:22.715548 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:22:22.715366 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b660b54b-5c60-483c-a01f-04a20247bc7f-offline-tls\") pod \"b660b54b-5c60-483c-a01f-04a20247bc7f\" (UID: \"b660b54b-5c60-483c-a01f-04a20247bc7f\") " Sep 08 10:22:22.715548 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:22:22.715427 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ns56p\" (UniqueName: \"kubernetes.io/projected/b660b54b-5c60-483c-a01f-04a20247bc7f-kube-api-access-ns56p\") pod \"b660b54b-5c60-483c-a01f-04a20247bc7f\" (UID: \"b660b54b-5c60-483c-a01f-04a20247bc7f\") " Sep 08 10:22:22.715548 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:22:22.715461 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b660b54b-5c60-483c-a01f-04a20247bc7f-registry-tls\") pod \"b660b54b-5c60-483c-a01f-04a20247bc7f\" (UID: \"b660b54b-5c60-483c-a01f-04a20247bc7f\") " Sep 08 10:22:22.715548 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:22:22.715497 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b660b54b-5c60-483c-a01f-04a20247bc7f-online-tls\") pod \"b660b54b-5c60-483c-a01f-04a20247bc7f\" (UID: \"b660b54b-5c60-483c-a01f-04a20247bc7f\") " Sep 08 10:22:22.717805 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:22:22.717770 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b660b54b-5c60-483c-a01f-04a20247bc7f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "b660b54b-5c60-483c-a01f-04a20247bc7f" (UID: "b660b54b-5c60-483c-a01f-04a20247bc7f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:22:22.717805 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:22:22.717785 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b660b54b-5c60-483c-a01f-04a20247bc7f-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "b660b54b-5c60-483c-a01f-04a20247bc7f" (UID: "b660b54b-5c60-483c-a01f-04a20247bc7f"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:22:22.718146 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:22:22.717806 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b660b54b-5c60-483c-a01f-04a20247bc7f-online-tls" (OuterVolumeSpecName: "online-tls") pod "b660b54b-5c60-483c-a01f-04a20247bc7f" (UID: "b660b54b-5c60-483c-a01f-04a20247bc7f"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:22:22.718146 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:22:22.717921 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b660b54b-5c60-483c-a01f-04a20247bc7f-kube-api-access-ns56p" (OuterVolumeSpecName: "kube-api-access-ns56p") pod "b660b54b-5c60-483c-a01f-04a20247bc7f" (UID: "b660b54b-5c60-483c-a01f-04a20247bc7f"). InnerVolumeSpecName "kube-api-access-ns56p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 10:22:22.723848 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:22:22.723826 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b660b54b-5c60-483c-a01f-04a20247bc7f-feast-data" (OuterVolumeSpecName: "feast-data") pod "b660b54b-5c60-483c-a01f-04a20247bc7f" (UID: "b660b54b-5c60-483c-a01f-04a20247bc7f"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 10:22:22.816628 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:22:22.816578 2568 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b660b54b-5c60-483c-a01f-04a20247bc7f-online-tls\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:22:22.816628 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:22:22.816599 2568 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b660b54b-5c60-483c-a01f-04a20247bc7f-feast-data\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:22:22.816628 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:22:22.816607 2568 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b660b54b-5c60-483c-a01f-04a20247bc7f-offline-tls\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:22:22.816628 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:22:22.816614 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ns56p\" (UniqueName: \"kubernetes.io/projected/b660b54b-5c60-483c-a01f-04a20247bc7f-kube-api-access-ns56p\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:22:22.816628 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:22:22.816624 2568 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b660b54b-5c60-483c-a01f-04a20247bc7f-registry-tls\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:22:23.127719 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:22:23.127690 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" event={"ID":"b660b54b-5c60-483c-a01f-04a20247bc7f","Type":"ContainerDied","Data":"05b176e69a601f2f14619b411bb83a01bd3db6239e76c399075f87aa3437b6ff"} Sep 08 10:22:23.127908 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:22:23.127728 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl" Sep 08 10:22:23.127908 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:22:23.127750 2568 scope.go:117] "RemoveContainer" containerID="f338919f7da33673e884c978a26b7b76651fec789cc5ef3fb4acd7b43a586678" Sep 08 10:22:23.136248 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:22:23.136233 2568 scope.go:117] "RemoveContainer" containerID="4021fbf21a9e02dbb48994bbb0c933c756de988b2bd7cf7cd1e74fbf62482192" Sep 08 10:22:23.144167 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:22:23.144152 2568 scope.go:117] "RemoveContainer" containerID="233b0a812abf389225575b44d461ba13786b3f95dd6677c6f39b486f85882b72" Sep 08 10:22:23.151288 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:22:23.151268 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl"] Sep 08 10:22:23.152004 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:22:23.151990 2568 scope.go:117] "RemoveContainer" containerID="ec27010a1662154ecf211a9b7ae8523359f0d85614224aaaddc1c22a8e03d6a6" Sep 08 10:22:23.159150 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:22:23.159130 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-credit-scoring-76868fd94f-zrbvl"] Sep 08 10:22:23.160261 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:22:23.160242 2568 scope.go:117] "RemoveContainer" containerID="2957c34ffcbcb1a578b5d33cf497a4c0a718d7de826519b68ead54807cb05079" Sep 08 10:22:24.403396 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:22:24.403369 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b660b54b-5c60-483c-a01f-04a20247bc7f" path="/var/lib/kubelet/pods/b660b54b-5c60-483c-a01f-04a20247bc7f/volumes" Sep 08 10:24:40.913762 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:40.913726 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-bmfg2"] Sep 08 10:24:40.914440 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:40.914031 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5d3466f-c146-4965-9f26-880204265c7b" containerName="postgres" Sep 08 10:24:40.914440 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:40.914046 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5d3466f-c146-4965-9f26-880204265c7b" containerName="postgres" Sep 08 10:24:40.914440 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:40.914057 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b660b54b-5c60-483c-a01f-04a20247bc7f" containerName="feast-apply" Sep 08 10:24:40.914440 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:40.914063 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b660b54b-5c60-483c-a01f-04a20247bc7f" containerName="feast-apply" Sep 08 10:24:40.914440 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:40.914072 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b660b54b-5c60-483c-a01f-04a20247bc7f" containerName="feast-init" Sep 08 10:24:40.914440 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:40.914077 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b660b54b-5c60-483c-a01f-04a20247bc7f" containerName="feast-init" Sep 08 10:24:40.914440 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:40.914086 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b660b54b-5c60-483c-a01f-04a20247bc7f" containerName="offline" Sep 08 10:24:40.914440 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:40.914092 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b660b54b-5c60-483c-a01f-04a20247bc7f" containerName="offline" Sep 08 10:24:40.914440 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:40.914099 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b660b54b-5c60-483c-a01f-04a20247bc7f" containerName="registry" Sep 08 10:24:40.914440 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:40.914106 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b660b54b-5c60-483c-a01f-04a20247bc7f" containerName="registry" Sep 08 10:24:40.914440 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:40.914128 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b660b54b-5c60-483c-a01f-04a20247bc7f" containerName="online" Sep 08 10:24:40.914440 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:40.914133 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b660b54b-5c60-483c-a01f-04a20247bc7f" containerName="online" Sep 08 10:24:40.914440 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:40.914180 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b660b54b-5c60-483c-a01f-04a20247bc7f" containerName="registry" Sep 08 10:24:40.914440 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:40.914190 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b660b54b-5c60-483c-a01f-04a20247bc7f" containerName="offline" Sep 08 10:24:40.914440 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:40.914200 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5d3466f-c146-4965-9f26-880204265c7b" containerName="postgres" Sep 08 10:24:40.914440 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:40.914209 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b660b54b-5c60-483c-a01f-04a20247bc7f" containerName="online" Sep 08 10:24:40.917141 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:40.917124 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-bmfg2" Sep 08 10:24:40.919636 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:40.919612 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\"" Sep 08 10:24:40.920552 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:40.920536 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\"" Sep 08 10:24:40.920622 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:40.920536 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-zmcx7\"" Sep 08 10:24:40.939350 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:40.939330 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-bmfg2"] Sep 08 10:24:41.044725 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:41.044700 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rl96f\" (UniqueName: \"kubernetes.io/projected/83dcef83-236e-42af-bf8c-20d44fb5bfe0-kube-api-access-rl96f\") pod \"redis-69bbd798b6-bmfg2\" (UID: \"83dcef83-236e-42af-bf8c-20d44fb5bfe0\") " pod="test-ns-feast-rest/redis-69bbd798b6-bmfg2" Sep 08 10:24:41.145920 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:41.145889 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl96f\" (UniqueName: \"kubernetes.io/projected/83dcef83-236e-42af-bf8c-20d44fb5bfe0-kube-api-access-rl96f\") pod \"redis-69bbd798b6-bmfg2\" (UID: \"83dcef83-236e-42af-bf8c-20d44fb5bfe0\") " pod="test-ns-feast-rest/redis-69bbd798b6-bmfg2" Sep 08 10:24:41.156214 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:41.156193 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rl96f\" (UniqueName: \"kubernetes.io/projected/83dcef83-236e-42af-bf8c-20d44fb5bfe0-kube-api-access-rl96f\") pod \"redis-69bbd798b6-bmfg2\" (UID: \"83dcef83-236e-42af-bf8c-20d44fb5bfe0\") " pod="test-ns-feast-rest/redis-69bbd798b6-bmfg2" Sep 08 10:24:41.241269 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:41.241217 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-bmfg2" Sep 08 10:24:41.508684 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:41.508662 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-bmfg2"] Sep 08 10:24:41.510724 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:24:41.510698 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod83dcef83_236e_42af_bf8c_20d44fb5bfe0.slice/crio-9f44058795dabfeb335e3ac8550ef77976b87f22ee09c6b42760280049a8bae5 WatchSource:0}: Error finding container 9f44058795dabfeb335e3ac8550ef77976b87f22ee09c6b42760280049a8bae5: Status 404 returned error can't find the container with id 9f44058795dabfeb335e3ac8550ef77976b87f22ee09c6b42760280049a8bae5 Sep 08 10:24:41.629292 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:41.629255 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-bmfg2" event={"ID":"83dcef83-236e-42af-bf8c-20d44fb5bfe0","Type":"ContainerStarted","Data":"9f44058795dabfeb335e3ac8550ef77976b87f22ee09c6b42760280049a8bae5"} Sep 08 10:24:45.645194 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:45.645158 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-bmfg2" event={"ID":"83dcef83-236e-42af-bf8c-20d44fb5bfe0","Type":"ContainerStarted","Data":"abef05d4d39e1548e756115c39e8395e71e0dae36ae1dd8f1b1704f6c0f06e6e"} Sep 08 10:24:45.659650 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:45.659608 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/redis-69bbd798b6-bmfg2" podStartSLOduration=2.4848918859999998 podStartE2EDuration="5.659593992s" totalImagesPullingTime="3.174702106s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:24:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:24:45.658914859 +0000 UTC m=+1069.703018926" watchObservedRunningTime="2026-09-08 10:24:45.659593992 +0000 UTC m=+1069.703698059" Sep 08 10:24:46.385193 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:46.385160 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-xmhr8"] Sep 08 10:24:46.388317 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:46.388295 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-xmhr8" Sep 08 10:24:46.390863 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:46.390842 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\"" Sep 08 10:24:46.395516 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:46.395471 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-xmhr8"] Sep 08 10:24:46.479461 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:46.479432 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/b22ba9b6-0432-4e28-b72f-2176538d22ed-postgresdata\") pod \"postgres-864846d4f5-xmhr8\" (UID: \"b22ba9b6-0432-4e28-b72f-2176538d22ed\") " pod="test-ns-feast-rest/postgres-864846d4f5-xmhr8" Sep 08 10:24:46.479577 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:46.479466 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nt9l7\" (UniqueName: \"kubernetes.io/projected/b22ba9b6-0432-4e28-b72f-2176538d22ed-kube-api-access-nt9l7\") pod \"postgres-864846d4f5-xmhr8\" (UID: \"b22ba9b6-0432-4e28-b72f-2176538d22ed\") " pod="test-ns-feast-rest/postgres-864846d4f5-xmhr8" Sep 08 10:24:46.580704 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:46.580680 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/b22ba9b6-0432-4e28-b72f-2176538d22ed-postgresdata\") pod \"postgres-864846d4f5-xmhr8\" (UID: \"b22ba9b6-0432-4e28-b72f-2176538d22ed\") " pod="test-ns-feast-rest/postgres-864846d4f5-xmhr8" Sep 08 10:24:46.580793 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:46.580710 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nt9l7\" (UniqueName: \"kubernetes.io/projected/b22ba9b6-0432-4e28-b72f-2176538d22ed-kube-api-access-nt9l7\") pod \"postgres-864846d4f5-xmhr8\" (UID: \"b22ba9b6-0432-4e28-b72f-2176538d22ed\") " pod="test-ns-feast-rest/postgres-864846d4f5-xmhr8" Sep 08 10:24:46.581078 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:46.581061 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/b22ba9b6-0432-4e28-b72f-2176538d22ed-postgresdata\") pod \"postgres-864846d4f5-xmhr8\" (UID: \"b22ba9b6-0432-4e28-b72f-2176538d22ed\") " pod="test-ns-feast-rest/postgres-864846d4f5-xmhr8" Sep 08 10:24:46.589022 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:46.588999 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nt9l7\" (UniqueName: \"kubernetes.io/projected/b22ba9b6-0432-4e28-b72f-2176538d22ed-kube-api-access-nt9l7\") pod \"postgres-864846d4f5-xmhr8\" (UID: \"b22ba9b6-0432-4e28-b72f-2176538d22ed\") " pod="test-ns-feast-rest/postgres-864846d4f5-xmhr8" Sep 08 10:24:46.699891 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:46.699824 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-xmhr8" Sep 08 10:24:46.966359 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:46.966205 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-xmhr8"] Sep 08 10:24:46.968799 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:24:46.968778 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb22ba9b6_0432_4e28_b72f_2176538d22ed.slice/crio-1394a68aaf1f68ad12c3f876c66ba9ed2eea85dd55e49935d1bc50b3b32d8f3e WatchSource:0}: Error finding container 1394a68aaf1f68ad12c3f876c66ba9ed2eea85dd55e49935d1bc50b3b32d8f3e: Status 404 returned error can't find the container with id 1394a68aaf1f68ad12c3f876c66ba9ed2eea85dd55e49935d1bc50b3b32d8f3e Sep 08 10:24:47.653452 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:47.653394 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-xmhr8" event={"ID":"b22ba9b6-0432-4e28-b72f-2176538d22ed","Type":"ContainerStarted","Data":"80167ff17d74561c1c6d35bb317c62857ed3039b6e02a35a6ba278e60cd6dd92"} Sep 08 10:24:47.653614 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:47.653458 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-xmhr8" event={"ID":"b22ba9b6-0432-4e28-b72f-2176538d22ed","Type":"ContainerStarted","Data":"1394a68aaf1f68ad12c3f876c66ba9ed2eea85dd55e49935d1bc50b3b32d8f3e"} Sep 08 10:24:47.670617 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:47.670565 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/postgres-864846d4f5-xmhr8" podStartSLOduration=1.670548567 podStartE2EDuration="1.670548567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:24:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:24:47.669533437 +0000 UTC m=+1071.713637526" watchObservedRunningTime="2026-09-08 10:24:47.670548567 +0000 UTC m=+1071.714652684" Sep 08 10:24:51.990822 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:51.990792 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v"] Sep 08 10:24:51.994755 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:51.994735 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" Sep 08 10:24:51.997168 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:51.997129 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-registry-tls\"" Sep 08 10:24:51.997168 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:51.997162 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-online-tls\"" Sep 08 10:24:51.997337 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:51.997177 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-dockercfg-fg5sh\"" Sep 08 10:24:51.997467 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:51.997453 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\"" Sep 08 10:24:52.004681 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:52.004661 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v"] Sep 08 10:24:52.124134 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:52.124107 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwb79\" (UniqueName: \"kubernetes.io/projected/db5968bb-1d95-4c73-8521-03415245056e-kube-api-access-pwb79\") pod \"feast-credit-scoring-9d67f7657-5xr2v\" (UID: \"db5968bb-1d95-4c73-8521-03415245056e\") " pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" Sep 08 10:24:52.124134 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:52.124138 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/db5968bb-1d95-4c73-8521-03415245056e-online-tls\") pod \"feast-credit-scoring-9d67f7657-5xr2v\" (UID: \"db5968bb-1d95-4c73-8521-03415245056e\") " pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" Sep 08 10:24:52.124314 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:52.124156 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/db5968bb-1d95-4c73-8521-03415245056e-registry-tls\") pod \"feast-credit-scoring-9d67f7657-5xr2v\" (UID: \"db5968bb-1d95-4c73-8521-03415245056e\") " pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" Sep 08 10:24:52.124314 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:52.124176 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/db5968bb-1d95-4c73-8521-03415245056e-feast-data\") pod \"feast-credit-scoring-9d67f7657-5xr2v\" (UID: \"db5968bb-1d95-4c73-8521-03415245056e\") " pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" Sep 08 10:24:52.224518 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:52.224492 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwb79\" (UniqueName: \"kubernetes.io/projected/db5968bb-1d95-4c73-8521-03415245056e-kube-api-access-pwb79\") pod \"feast-credit-scoring-9d67f7657-5xr2v\" (UID: \"db5968bb-1d95-4c73-8521-03415245056e\") " pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" Sep 08 10:24:52.224518 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:52.224519 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/db5968bb-1d95-4c73-8521-03415245056e-online-tls\") pod \"feast-credit-scoring-9d67f7657-5xr2v\" (UID: \"db5968bb-1d95-4c73-8521-03415245056e\") " pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" Sep 08 10:24:52.224718 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:52.224538 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/db5968bb-1d95-4c73-8521-03415245056e-registry-tls\") pod \"feast-credit-scoring-9d67f7657-5xr2v\" (UID: \"db5968bb-1d95-4c73-8521-03415245056e\") " pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" Sep 08 10:24:52.224718 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:52.224558 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/db5968bb-1d95-4c73-8521-03415245056e-feast-data\") pod \"feast-credit-scoring-9d67f7657-5xr2v\" (UID: \"db5968bb-1d95-4c73-8521-03415245056e\") " pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" Sep 08 10:24:52.225015 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:52.224994 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/db5968bb-1d95-4c73-8521-03415245056e-feast-data\") pod \"feast-credit-scoring-9d67f7657-5xr2v\" (UID: \"db5968bb-1d95-4c73-8521-03415245056e\") " pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" Sep 08 10:24:52.227014 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:52.226994 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/db5968bb-1d95-4c73-8521-03415245056e-online-tls\") pod \"feast-credit-scoring-9d67f7657-5xr2v\" (UID: \"db5968bb-1d95-4c73-8521-03415245056e\") " pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" Sep 08 10:24:52.227118 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:52.227102 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/db5968bb-1d95-4c73-8521-03415245056e-registry-tls\") pod \"feast-credit-scoring-9d67f7657-5xr2v\" (UID: \"db5968bb-1d95-4c73-8521-03415245056e\") " pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" Sep 08 10:24:52.232933 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:52.232914 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwb79\" (UniqueName: \"kubernetes.io/projected/db5968bb-1d95-4c73-8521-03415245056e-kube-api-access-pwb79\") pod \"feast-credit-scoring-9d67f7657-5xr2v\" (UID: \"db5968bb-1d95-4c73-8521-03415245056e\") " pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" Sep 08 10:24:52.304197 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:52.304142 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" Sep 08 10:24:52.578385 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:52.578360 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v"] Sep 08 10:24:52.580639 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:24:52.580615 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb5968bb_1d95_4c73_8521_03415245056e.slice/crio-18c82aa47a0c702f3e4fa11a985e7cdebba78539fc8b142cc9b5d8d586789fda WatchSource:0}: Error finding container 18c82aa47a0c702f3e4fa11a985e7cdebba78539fc8b142cc9b5d8d586789fda: Status 404 returned error can't find the container with id 18c82aa47a0c702f3e4fa11a985e7cdebba78539fc8b142cc9b5d8d586789fda Sep 08 10:24:52.672797 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:52.672771 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" event={"ID":"db5968bb-1d95-4c73-8521-03415245056e","Type":"ContainerStarted","Data":"965c2ee6b5648e75747e65c803be89ea3ceabef7869bae0ee64622f6139b2900"} Sep 08 10:24:52.672913 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:52.672819 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" event={"ID":"db5968bb-1d95-4c73-8521-03415245056e","Type":"ContainerStarted","Data":"18c82aa47a0c702f3e4fa11a985e7cdebba78539fc8b142cc9b5d8d586789fda"} Sep 08 10:24:53.678192 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:53.678157 2568 generic.go:356] "Generic (PLEG): container finished" podID="db5968bb-1d95-4c73-8521-03415245056e" containerID="965c2ee6b5648e75747e65c803be89ea3ceabef7869bae0ee64622f6139b2900" exitCode=0 Sep 08 10:24:53.678540 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:53.678241 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" event={"ID":"db5968bb-1d95-4c73-8521-03415245056e","Type":"ContainerDied","Data":"965c2ee6b5648e75747e65c803be89ea3ceabef7869bae0ee64622f6139b2900"} Sep 08 10:24:54.684491 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:24:54.684455 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" event={"ID":"db5968bb-1d95-4c73-8521-03415245056e","Type":"ContainerStarted","Data":"b8e61a6a4cc1189d833b8c196dc20244d1dc41c8846f0b9e48e2634d4da129af"} Sep 08 10:25:00.711261 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:00.711230 2568 generic.go:356] "Generic (PLEG): container finished" podID="db5968bb-1d95-4c73-8521-03415245056e" containerID="b8e61a6a4cc1189d833b8c196dc20244d1dc41c8846f0b9e48e2634d4da129af" exitCode=0 Sep 08 10:25:00.711585 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:00.711300 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" event={"ID":"db5968bb-1d95-4c73-8521-03415245056e","Type":"ContainerDied","Data":"b8e61a6a4cc1189d833b8c196dc20244d1dc41c8846f0b9e48e2634d4da129af"} Sep 08 10:25:00.715668 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:00.715653 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\"" Sep 08 10:25:01.719161 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:01.719127 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" event={"ID":"db5968bb-1d95-4c73-8521-03415245056e","Type":"ContainerStarted","Data":"7c6cbb87a3a401067af5c09641c6594a3231765e17d62ca3f47efdbd54cf3df8"} Sep 08 10:25:01.719550 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:01.719179 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" event={"ID":"db5968bb-1d95-4c73-8521-03415245056e","Type":"ContainerStarted","Data":"8570e6cb907a1d0273287f7003003ae5d234f65c203a58426db013774264e10f"} Sep 08 10:25:01.741818 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:01.741776 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" podStartSLOduration=10.741756927 podStartE2EDuration="10.741756927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:24:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:25:01.740131705 +0000 UTC m=+1085.784235799" watchObservedRunningTime="2026-09-08 10:25:01.741756927 +0000 UTC m=+1085.785861003" Sep 08 10:25:02.305116 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:02.305084 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" Sep 08 10:25:02.305255 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:02.305154 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" Sep 08 10:25:04.304425 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:04.304366 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" Sep 08 10:25:04.304887 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:04.304523 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" Sep 08 10:25:04.306137 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:04.306110 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" podUID="db5968bb-1d95-4c73-8521-03415245056e" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.32:6571: connect: connection refused" Sep 08 10:25:04.306249 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:04.306154 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" podUID="db5968bb-1d95-4c73-8521-03415245056e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused" Sep 08 10:25:07.305322 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:07.305278 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" podUID="db5968bb-1d95-4c73-8521-03415245056e" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.32:6571: connect: connection refused" Sep 08 10:25:07.311148 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:07.311118 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" Sep 08 10:25:07.314543 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:07.314521 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" Sep 08 10:25:10.304596 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:10.304545 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" podUID="db5968bb-1d95-4c73-8521-03415245056e" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.32:6571: connect: connection refused" Sep 08 10:25:13.305659 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:13.305629 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" Sep 08 10:25:13.306355 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:13.306338 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" Sep 08 10:25:17.971931 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:17.971901 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v"] Sep 08 10:25:17.978900 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:17.978876 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" Sep 08 10:25:17.982692 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:17.982670 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-registry-tls\"" Sep 08 10:25:17.982908 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:17.982880 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-dockercfg-rqqvc\"" Sep 08 10:25:17.984060 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:17.984046 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-online-tls\"" Sep 08 10:25:17.985210 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:17.985192 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\"" Sep 08 10:25:17.990916 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:17.990897 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v"] Sep 08 10:25:18.039204 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:18.039180 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b793b2e9-1e40-4e6c-96f5-a2c3b53fd982-online-tls\") pod \"feast-driver-ranking-7f487885db-dxs8v\" (UID: \"b793b2e9-1e40-4e6c-96f5-a2c3b53fd982\") " pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" Sep 08 10:25:18.039329 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:18.039213 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b793b2e9-1e40-4e6c-96f5-a2c3b53fd982-feast-data\") pod \"feast-driver-ranking-7f487885db-dxs8v\" (UID: \"b793b2e9-1e40-4e6c-96f5-a2c3b53fd982\") " pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" Sep 08 10:25:18.039329 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:18.039241 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9xf4\" (UniqueName: \"kubernetes.io/projected/b793b2e9-1e40-4e6c-96f5-a2c3b53fd982-kube-api-access-c9xf4\") pod \"feast-driver-ranking-7f487885db-dxs8v\" (UID: \"b793b2e9-1e40-4e6c-96f5-a2c3b53fd982\") " pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" Sep 08 10:25:18.039403 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:18.039344 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b793b2e9-1e40-4e6c-96f5-a2c3b53fd982-registry-tls\") pod \"feast-driver-ranking-7f487885db-dxs8v\" (UID: \"b793b2e9-1e40-4e6c-96f5-a2c3b53fd982\") " pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" Sep 08 10:25:18.140510 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:18.140484 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b793b2e9-1e40-4e6c-96f5-a2c3b53fd982-online-tls\") pod \"feast-driver-ranking-7f487885db-dxs8v\" (UID: \"b793b2e9-1e40-4e6c-96f5-a2c3b53fd982\") " pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" Sep 08 10:25:18.140615 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:18.140517 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b793b2e9-1e40-4e6c-96f5-a2c3b53fd982-feast-data\") pod \"feast-driver-ranking-7f487885db-dxs8v\" (UID: \"b793b2e9-1e40-4e6c-96f5-a2c3b53fd982\") " pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" Sep 08 10:25:18.140615 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:18.140542 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9xf4\" (UniqueName: \"kubernetes.io/projected/b793b2e9-1e40-4e6c-96f5-a2c3b53fd982-kube-api-access-c9xf4\") pod \"feast-driver-ranking-7f487885db-dxs8v\" (UID: \"b793b2e9-1e40-4e6c-96f5-a2c3b53fd982\") " pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" Sep 08 10:25:18.140615 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:18.140577 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b793b2e9-1e40-4e6c-96f5-a2c3b53fd982-registry-tls\") pod \"feast-driver-ranking-7f487885db-dxs8v\" (UID: \"b793b2e9-1e40-4e6c-96f5-a2c3b53fd982\") " pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" Sep 08 10:25:18.140962 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:18.140942 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b793b2e9-1e40-4e6c-96f5-a2c3b53fd982-feast-data\") pod \"feast-driver-ranking-7f487885db-dxs8v\" (UID: \"b793b2e9-1e40-4e6c-96f5-a2c3b53fd982\") " pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" Sep 08 10:25:18.143053 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:18.143032 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b793b2e9-1e40-4e6c-96f5-a2c3b53fd982-registry-tls\") pod \"feast-driver-ranking-7f487885db-dxs8v\" (UID: \"b793b2e9-1e40-4e6c-96f5-a2c3b53fd982\") " pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" Sep 08 10:25:18.143135 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:18.143053 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b793b2e9-1e40-4e6c-96f5-a2c3b53fd982-online-tls\") pod \"feast-driver-ranking-7f487885db-dxs8v\" (UID: \"b793b2e9-1e40-4e6c-96f5-a2c3b53fd982\") " pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" Sep 08 10:25:18.149294 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:18.149274 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9xf4\" (UniqueName: \"kubernetes.io/projected/b793b2e9-1e40-4e6c-96f5-a2c3b53fd982-kube-api-access-c9xf4\") pod \"feast-driver-ranking-7f487885db-dxs8v\" (UID: \"b793b2e9-1e40-4e6c-96f5-a2c3b53fd982\") " pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" Sep 08 10:25:18.287889 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:18.287822 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" Sep 08 10:25:18.561236 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:18.561214 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v"] Sep 08 10:25:18.563463 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:25:18.563436 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb793b2e9_1e40_4e6c_96f5_a2c3b53fd982.slice/crio-d575d24d0421fca8283b0a62e6accfa6279b5561f8486b7dc7a9a2d5b3dbd784 WatchSource:0}: Error finding container d575d24d0421fca8283b0a62e6accfa6279b5561f8486b7dc7a9a2d5b3dbd784: Status 404 returned error can't find the container with id d575d24d0421fca8283b0a62e6accfa6279b5561f8486b7dc7a9a2d5b3dbd784 Sep 08 10:25:18.798429 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:18.793152 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" event={"ID":"b793b2e9-1e40-4e6c-96f5-a2c3b53fd982","Type":"ContainerStarted","Data":"7ea3912ef8428acdb804417e1131a892386181f82924f8b3c00b9567091d839e"} Sep 08 10:25:18.798429 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:18.793214 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" event={"ID":"b793b2e9-1e40-4e6c-96f5-a2c3b53fd982","Type":"ContainerStarted","Data":"d575d24d0421fca8283b0a62e6accfa6279b5561f8486b7dc7a9a2d5b3dbd784"} Sep 08 10:25:23.814754 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:23.814723 2568 generic.go:356] "Generic (PLEG): container finished" podID="b793b2e9-1e40-4e6c-96f5-a2c3b53fd982" containerID="7ea3912ef8428acdb804417e1131a892386181f82924f8b3c00b9567091d839e" exitCode=0 Sep 08 10:25:23.815102 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:23.814796 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" event={"ID":"b793b2e9-1e40-4e6c-96f5-a2c3b53fd982","Type":"ContainerDied","Data":"7ea3912ef8428acdb804417e1131a892386181f82924f8b3c00b9567091d839e"} Sep 08 10:25:24.822108 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:24.822076 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" event={"ID":"b793b2e9-1e40-4e6c-96f5-a2c3b53fd982","Type":"ContainerStarted","Data":"dcd20c101e7ea0c274096d5e3731d93779a8545d0a07f491a2dda7c59785efad"} Sep 08 10:25:30.850219 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:30.850132 2568 generic.go:356] "Generic (PLEG): container finished" podID="b793b2e9-1e40-4e6c-96f5-a2c3b53fd982" containerID="dcd20c101e7ea0c274096d5e3731d93779a8545d0a07f491a2dda7c59785efad" exitCode=0 Sep 08 10:25:30.850219 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:30.850203 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" event={"ID":"b793b2e9-1e40-4e6c-96f5-a2c3b53fd982","Type":"ContainerDied","Data":"dcd20c101e7ea0c274096d5e3731d93779a8545d0a07f491a2dda7c59785efad"} Sep 08 10:25:31.858990 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:31.858952 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" event={"ID":"b793b2e9-1e40-4e6c-96f5-a2c3b53fd982","Type":"ContainerStarted","Data":"9cef30b951df666da65b0aea144162c45fbdf7dc987affd6259be4c3a3c40808"} Sep 08 10:25:31.859352 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:31.859025 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" event={"ID":"b793b2e9-1e40-4e6c-96f5-a2c3b53fd982","Type":"ContainerStarted","Data":"060187d913c69f180d96fd4c0282d287e8c96578e36351bb803032737305b37f"} Sep 08 10:25:31.885956 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:31.885906 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" podStartSLOduration=14.885888502 podStartE2EDuration="14.885888502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:25:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:25:31.884310832 +0000 UTC m=+1115.928414924" watchObservedRunningTime="2026-09-08 10:25:31.885888502 +0000 UTC m=+1115.929992570" Sep 08 10:25:33.288746 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:33.288703 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" Sep 08 10:25:33.289200 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:33.288786 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" Sep 08 10:25:33.290422 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:33.290368 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" podUID="b793b2e9-1e40-4e6c-96f5-a2c3b53fd982" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.33:6571: connect: connection refused" Sep 08 10:25:33.290568 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:33.290441 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" podUID="b793b2e9-1e40-4e6c-96f5-a2c3b53fd982" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused" Sep 08 10:25:36.289062 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:36.289020 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" podUID="b793b2e9-1e40-4e6c-96f5-a2c3b53fd982" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.33:6571: connect: connection refused" Sep 08 10:25:36.289489 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:36.289123 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" podUID="b793b2e9-1e40-4e6c-96f5-a2c3b53fd982" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused" Sep 08 10:25:38.288761 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:38.288709 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" Sep 08 10:25:38.289233 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:38.288790 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" Sep 08 10:25:39.288647 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:39.288601 2568 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" podUID="b793b2e9-1e40-4e6c-96f5-a2c3b53fd982" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.33:6571: connect: connection refused" Sep 08 10:25:39.296856 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:39.296831 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" Sep 08 10:25:39.903217 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:39.903185 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" Sep 08 10:25:42.289324 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:42.289297 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" Sep 08 10:25:42.290150 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:25:42.290128 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" Sep 08 10:27:00.618156 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:00.618126 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-794745947f-qkcfq"] Sep 08 10:27:00.618590 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:00.618337 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-794745947f-qkcfq" podUID="3bcb6d04-6fcf-467f-ae66-48a518ebb7b2" containerName="manager" containerID="cri-o://5d357a67c15a923906370a1d69d614218d8861e24d9bf6633de3193089ea251a" gracePeriod=10 Sep 08 10:27:00.873999 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:00.873930 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-794745947f-qkcfq" Sep 08 10:27:00.942936 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:00.942895 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rsqrd\" (UniqueName: \"kubernetes.io/projected/3bcb6d04-6fcf-467f-ae66-48a518ebb7b2-kube-api-access-rsqrd\") pod \"3bcb6d04-6fcf-467f-ae66-48a518ebb7b2\" (UID: \"3bcb6d04-6fcf-467f-ae66-48a518ebb7b2\") " Sep 08 10:27:00.945242 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:00.945209 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bcb6d04-6fcf-467f-ae66-48a518ebb7b2-kube-api-access-rsqrd" (OuterVolumeSpecName: "kube-api-access-rsqrd") pod "3bcb6d04-6fcf-467f-ae66-48a518ebb7b2" (UID: "3bcb6d04-6fcf-467f-ae66-48a518ebb7b2"). InnerVolumeSpecName "kube-api-access-rsqrd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 10:27:00.970750 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:00.970723 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v"] Sep 08 10:27:00.970964 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:00.970940 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" podUID="db5968bb-1d95-4c73-8521-03415245056e" containerName="registry" containerID="cri-o://8570e6cb907a1d0273287f7003003ae5d234f65c203a58426db013774264e10f" gracePeriod=30 Sep 08 10:27:00.971051 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:00.971002 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" podUID="db5968bb-1d95-4c73-8521-03415245056e" containerName="online" containerID="cri-o://7c6cbb87a3a401067af5c09641c6594a3231765e17d62ca3f47efdbd54cf3df8" gracePeriod=30 Sep 08 10:27:00.976286 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:00.976263 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v"] Sep 08 10:27:00.976526 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:00.976503 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" podUID="b793b2e9-1e40-4e6c-96f5-a2c3b53fd982" containerName="registry" containerID="cri-o://060187d913c69f180d96fd4c0282d287e8c96578e36351bb803032737305b37f" gracePeriod=30 Sep 08 10:27:00.976628 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:00.976521 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" podUID="b793b2e9-1e40-4e6c-96f5-a2c3b53fd982" containerName="online" containerID="cri-o://9cef30b951df666da65b0aea144162c45fbdf7dc987affd6259be4c3a3c40808" gracePeriod=30 Sep 08 10:27:01.043786 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:01.043760 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rsqrd\" (UniqueName: \"kubernetes.io/projected/3bcb6d04-6fcf-467f-ae66-48a518ebb7b2-kube-api-access-rsqrd\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:27:01.241719 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:01.241639 2568 generic.go:356] "Generic (PLEG): container finished" podID="3bcb6d04-6fcf-467f-ae66-48a518ebb7b2" containerID="5d357a67c15a923906370a1d69d614218d8861e24d9bf6633de3193089ea251a" exitCode=0 Sep 08 10:27:01.241719 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:01.241684 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-794745947f-qkcfq" Sep 08 10:27:01.241897 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:01.241722 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-794745947f-qkcfq" event={"ID":"3bcb6d04-6fcf-467f-ae66-48a518ebb7b2","Type":"ContainerDied","Data":"5d357a67c15a923906370a1d69d614218d8861e24d9bf6633de3193089ea251a"} Sep 08 10:27:01.241897 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:01.241780 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-794745947f-qkcfq" event={"ID":"3bcb6d04-6fcf-467f-ae66-48a518ebb7b2","Type":"ContainerDied","Data":"6ac95d2418ea7d2a776467b57168556607b33856074592c1cdb075fac831f805"} Sep 08 10:27:01.241897 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:01.241802 2568 scope.go:117] "RemoveContainer" containerID="5d357a67c15a923906370a1d69d614218d8861e24d9bf6633de3193089ea251a" Sep 08 10:27:01.250546 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:01.250530 2568 scope.go:117] "RemoveContainer" containerID="5d357a67c15a923906370a1d69d614218d8861e24d9bf6633de3193089ea251a" Sep 08 10:27:01.250827 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:27:01.250808 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d357a67c15a923906370a1d69d614218d8861e24d9bf6633de3193089ea251a\": container with ID starting with 5d357a67c15a923906370a1d69d614218d8861e24d9bf6633de3193089ea251a not found: ID does not exist" containerID="5d357a67c15a923906370a1d69d614218d8861e24d9bf6633de3193089ea251a" Sep 08 10:27:01.250874 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:01.250834 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d357a67c15a923906370a1d69d614218d8861e24d9bf6633de3193089ea251a"} err="failed to get container status \"5d357a67c15a923906370a1d69d614218d8861e24d9bf6633de3193089ea251a\": rpc error: code = NotFound desc = could not find container \"5d357a67c15a923906370a1d69d614218d8861e24d9bf6633de3193089ea251a\": container with ID starting with 5d357a67c15a923906370a1d69d614218d8861e24d9bf6633de3193089ea251a not found: ID does not exist" Sep 08 10:27:01.262801 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:01.262776 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-794745947f-qkcfq"] Sep 08 10:27:01.266498 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:01.266476 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-794745947f-qkcfq"] Sep 08 10:27:02.248309 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:02.248279 2568 generic.go:356] "Generic (PLEG): container finished" podID="b793b2e9-1e40-4e6c-96f5-a2c3b53fd982" containerID="9cef30b951df666da65b0aea144162c45fbdf7dc987affd6259be4c3a3c40808" exitCode=0 Sep 08 10:27:02.248712 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:02.248349 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" event={"ID":"b793b2e9-1e40-4e6c-96f5-a2c3b53fd982","Type":"ContainerDied","Data":"9cef30b951df666da65b0aea144162c45fbdf7dc987affd6259be4c3a3c40808"} Sep 08 10:27:02.250646 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:02.250624 2568 generic.go:356] "Generic (PLEG): container finished" podID="db5968bb-1d95-4c73-8521-03415245056e" containerID="7c6cbb87a3a401067af5c09641c6594a3231765e17d62ca3f47efdbd54cf3df8" exitCode=0 Sep 08 10:27:02.250759 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:02.250656 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" event={"ID":"db5968bb-1d95-4c73-8521-03415245056e","Type":"ContainerDied","Data":"7c6cbb87a3a401067af5c09641c6594a3231765e17d62ca3f47efdbd54cf3df8"} Sep 08 10:27:02.404054 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:02.404014 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3bcb6d04-6fcf-467f-ae66-48a518ebb7b2" path="/var/lib/kubelet/pods/3bcb6d04-6fcf-467f-ae66-48a518ebb7b2/volumes" Sep 08 10:27:03.117960 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:03.117925 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-xmhr8"] Sep 08 10:27:03.118128 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:03.118094 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/postgres-864846d4f5-xmhr8" podUID="b22ba9b6-0432-4e28-b72f-2176538d22ed" containerName="postgres" containerID="cri-o://80167ff17d74561c1c6d35bb317c62857ed3039b6e02a35a6ba278e60cd6dd92" gracePeriod=30 Sep 08 10:27:03.128807 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:03.128779 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-bmfg2"] Sep 08 10:27:03.128947 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:03.128930 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/redis-69bbd798b6-bmfg2" podUID="83dcef83-236e-42af-bf8c-20d44fb5bfe0" containerName="redis" containerID="cri-o://abef05d4d39e1548e756115c39e8395e71e0dae36ae1dd8f1b1704f6c0f06e6e" gracePeriod=30 Sep 08 10:27:03.256349 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:03.256325 2568 generic.go:356] "Generic (PLEG): container finished" podID="b22ba9b6-0432-4e28-b72f-2176538d22ed" containerID="80167ff17d74561c1c6d35bb317c62857ed3039b6e02a35a6ba278e60cd6dd92" exitCode=0 Sep 08 10:27:03.256693 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:03.256400 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-xmhr8" event={"ID":"b22ba9b6-0432-4e28-b72f-2176538d22ed","Type":"ContainerDied","Data":"80167ff17d74561c1c6d35bb317c62857ed3039b6e02a35a6ba278e60cd6dd92"} Sep 08 10:27:03.258111 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:03.258088 2568 generic.go:356] "Generic (PLEG): container finished" podID="83dcef83-236e-42af-bf8c-20d44fb5bfe0" containerID="abef05d4d39e1548e756115c39e8395e71e0dae36ae1dd8f1b1704f6c0f06e6e" exitCode=0 Sep 08 10:27:03.258213 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:03.258161 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-bmfg2" event={"ID":"83dcef83-236e-42af-bf8c-20d44fb5bfe0","Type":"ContainerDied","Data":"abef05d4d39e1548e756115c39e8395e71e0dae36ae1dd8f1b1704f6c0f06e6e"} Sep 08 10:27:03.385711 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:03.385689 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-bmfg2" Sep 08 10:27:03.388452 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:03.388435 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-xmhr8" Sep 08 10:27:03.462865 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:03.462838 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rl96f\" (UniqueName: \"kubernetes.io/projected/83dcef83-236e-42af-bf8c-20d44fb5bfe0-kube-api-access-rl96f\") pod \"83dcef83-236e-42af-bf8c-20d44fb5bfe0\" (UID: \"83dcef83-236e-42af-bf8c-20d44fb5bfe0\") " Sep 08 10:27:03.463041 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:03.462955 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/b22ba9b6-0432-4e28-b72f-2176538d22ed-postgresdata\") pod \"b22ba9b6-0432-4e28-b72f-2176538d22ed\" (UID: \"b22ba9b6-0432-4e28-b72f-2176538d22ed\") " Sep 08 10:27:03.463041 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:03.462996 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nt9l7\" (UniqueName: \"kubernetes.io/projected/b22ba9b6-0432-4e28-b72f-2176538d22ed-kube-api-access-nt9l7\") pod \"b22ba9b6-0432-4e28-b72f-2176538d22ed\" (UID: \"b22ba9b6-0432-4e28-b72f-2176538d22ed\") " Sep 08 10:27:03.465240 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:03.465205 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b22ba9b6-0432-4e28-b72f-2176538d22ed-kube-api-access-nt9l7" (OuterVolumeSpecName: "kube-api-access-nt9l7") pod "b22ba9b6-0432-4e28-b72f-2176538d22ed" (UID: "b22ba9b6-0432-4e28-b72f-2176538d22ed"). InnerVolumeSpecName "kube-api-access-nt9l7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 10:27:03.465378 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:03.465361 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83dcef83-236e-42af-bf8c-20d44fb5bfe0-kube-api-access-rl96f" (OuterVolumeSpecName: "kube-api-access-rl96f") pod "83dcef83-236e-42af-bf8c-20d44fb5bfe0" (UID: "83dcef83-236e-42af-bf8c-20d44fb5bfe0"). InnerVolumeSpecName "kube-api-access-rl96f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 10:27:03.493321 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:03.493295 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b22ba9b6-0432-4e28-b72f-2176538d22ed-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "b22ba9b6-0432-4e28-b72f-2176538d22ed" (UID: "b22ba9b6-0432-4e28-b72f-2176538d22ed"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 10:27:03.563811 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:03.563777 2568 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/b22ba9b6-0432-4e28-b72f-2176538d22ed-postgresdata\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:27:03.563811 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:03.563809 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nt9l7\" (UniqueName: \"kubernetes.io/projected/b22ba9b6-0432-4e28-b72f-2176538d22ed-kube-api-access-nt9l7\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:27:03.563811 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:03.563820 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rl96f\" (UniqueName: \"kubernetes.io/projected/83dcef83-236e-42af-bf8c-20d44fb5bfe0-kube-api-access-rl96f\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:27:04.262711 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:04.262692 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-bmfg2" Sep 08 10:27:04.263101 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:04.262727 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-bmfg2" event={"ID":"83dcef83-236e-42af-bf8c-20d44fb5bfe0","Type":"ContainerDied","Data":"9f44058795dabfeb335e3ac8550ef77976b87f22ee09c6b42760280049a8bae5"} Sep 08 10:27:04.263101 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:04.262773 2568 scope.go:117] "RemoveContainer" containerID="abef05d4d39e1548e756115c39e8395e71e0dae36ae1dd8f1b1704f6c0f06e6e" Sep 08 10:27:04.264097 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:04.264064 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-xmhr8" event={"ID":"b22ba9b6-0432-4e28-b72f-2176538d22ed","Type":"ContainerDied","Data":"1394a68aaf1f68ad12c3f876c66ba9ed2eea85dd55e49935d1bc50b3b32d8f3e"} Sep 08 10:27:04.264160 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:04.264096 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-xmhr8" Sep 08 10:27:04.271447 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:04.271428 2568 scope.go:117] "RemoveContainer" containerID="80167ff17d74561c1c6d35bb317c62857ed3039b6e02a35a6ba278e60cd6dd92" Sep 08 10:27:04.285588 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:04.285568 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-bmfg2"] Sep 08 10:27:04.290639 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:04.290617 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-bmfg2"] Sep 08 10:27:04.300724 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:04.300703 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-xmhr8"] Sep 08 10:27:04.307055 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:04.307036 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-xmhr8"] Sep 08 10:27:04.403038 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:04.403019 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83dcef83-236e-42af-bf8c-20d44fb5bfe0" path="/var/lib/kubelet/pods/83dcef83-236e-42af-bf8c-20d44fb5bfe0/volumes" Sep 08 10:27:04.403330 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:04.403318 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b22ba9b6-0432-4e28-b72f-2176538d22ed" path="/var/lib/kubelet/pods/b22ba9b6-0432-4e28-b72f-2176538d22ed/volumes" Sep 08 10:27:07.312525 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:07.312493 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" podUID="db5968bb-1d95-4c73-8521-03415245056e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused" Sep 08 10:27:09.900741 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:09.900710 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" podUID="b793b2e9-1e40-4e6c-96f5-a2c3b53fd982" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused" Sep 08 10:27:17.312017 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:17.311980 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" podUID="db5968bb-1d95-4c73-8521-03415245056e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused" Sep 08 10:27:19.900594 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:19.900556 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" podUID="b793b2e9-1e40-4e6c-96f5-a2c3b53fd982" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused" Sep 08 10:27:27.311778 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:27.311731 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" podUID="db5968bb-1d95-4c73-8521-03415245056e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused" Sep 08 10:27:27.312278 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:27.311856 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" Sep 08 10:27:29.900577 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:29.900538 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" podUID="b793b2e9-1e40-4e6c-96f5-a2c3b53fd982" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused" Sep 08 10:27:29.900942 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:29.900664 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" Sep 08 10:27:31.372701 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:31.372676 2568 generic.go:356] "Generic (PLEG): container finished" podID="b793b2e9-1e40-4e6c-96f5-a2c3b53fd982" containerID="060187d913c69f180d96fd4c0282d287e8c96578e36351bb803032737305b37f" exitCode=137 Sep 08 10:27:31.373063 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:31.372748 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" event={"ID":"b793b2e9-1e40-4e6c-96f5-a2c3b53fd982","Type":"ContainerDied","Data":"060187d913c69f180d96fd4c0282d287e8c96578e36351bb803032737305b37f"} Sep 08 10:27:31.375056 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:31.375034 2568 generic.go:356] "Generic (PLEG): container finished" podID="db5968bb-1d95-4c73-8521-03415245056e" containerID="8570e6cb907a1d0273287f7003003ae5d234f65c203a58426db013774264e10f" exitCode=137 Sep 08 10:27:31.375165 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:31.375098 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" event={"ID":"db5968bb-1d95-4c73-8521-03415245056e","Type":"ContainerDied","Data":"8570e6cb907a1d0273287f7003003ae5d234f65c203a58426db013774264e10f"} Sep 08 10:27:31.629004 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:31.628955 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" Sep 08 10:27:31.631542 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:31.631526 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" Sep 08 10:27:31.758328 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:31.758290 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b793b2e9-1e40-4e6c-96f5-a2c3b53fd982-online-tls\") pod \"b793b2e9-1e40-4e6c-96f5-a2c3b53fd982\" (UID: \"b793b2e9-1e40-4e6c-96f5-a2c3b53fd982\") " Sep 08 10:27:31.758499 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:31.758353 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b793b2e9-1e40-4e6c-96f5-a2c3b53fd982-feast-data\") pod \"b793b2e9-1e40-4e6c-96f5-a2c3b53fd982\" (UID: \"b793b2e9-1e40-4e6c-96f5-a2c3b53fd982\") " Sep 08 10:27:31.758499 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:31.758373 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c9xf4\" (UniqueName: \"kubernetes.io/projected/b793b2e9-1e40-4e6c-96f5-a2c3b53fd982-kube-api-access-c9xf4\") pod \"b793b2e9-1e40-4e6c-96f5-a2c3b53fd982\" (UID: \"b793b2e9-1e40-4e6c-96f5-a2c3b53fd982\") " Sep 08 10:27:31.758499 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:31.758477 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b793b2e9-1e40-4e6c-96f5-a2c3b53fd982-registry-tls\") pod \"b793b2e9-1e40-4e6c-96f5-a2c3b53fd982\" (UID: \"b793b2e9-1e40-4e6c-96f5-a2c3b53fd982\") " Sep 08 10:27:31.758666 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:31.758512 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pwb79\" (UniqueName: \"kubernetes.io/projected/db5968bb-1d95-4c73-8521-03415245056e-kube-api-access-pwb79\") pod \"db5968bb-1d95-4c73-8521-03415245056e\" (UID: \"db5968bb-1d95-4c73-8521-03415245056e\") " Sep 08 10:27:31.758666 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:31.758535 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/db5968bb-1d95-4c73-8521-03415245056e-feast-data\") pod \"db5968bb-1d95-4c73-8521-03415245056e\" (UID: \"db5968bb-1d95-4c73-8521-03415245056e\") " Sep 08 10:27:31.758666 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:31.758579 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/db5968bb-1d95-4c73-8521-03415245056e-online-tls\") pod \"db5968bb-1d95-4c73-8521-03415245056e\" (UID: \"db5968bb-1d95-4c73-8521-03415245056e\") " Sep 08 10:27:31.758666 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:31.758647 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/db5968bb-1d95-4c73-8521-03415245056e-registry-tls\") pod \"db5968bb-1d95-4c73-8521-03415245056e\" (UID: \"db5968bb-1d95-4c73-8521-03415245056e\") " Sep 08 10:27:31.759016 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:31.758964 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b793b2e9-1e40-4e6c-96f5-a2c3b53fd982-feast-data" (OuterVolumeSpecName: "feast-data") pod "b793b2e9-1e40-4e6c-96f5-a2c3b53fd982" (UID: "b793b2e9-1e40-4e6c-96f5-a2c3b53fd982"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 10:27:31.761044 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:31.761014 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db5968bb-1d95-4c73-8521-03415245056e-online-tls" (OuterVolumeSpecName: "online-tls") pod "db5968bb-1d95-4c73-8521-03415245056e" (UID: "db5968bb-1d95-4c73-8521-03415245056e"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:27:31.761240 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:31.761208 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db5968bb-1d95-4c73-8521-03415245056e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "db5968bb-1d95-4c73-8521-03415245056e" (UID: "db5968bb-1d95-4c73-8521-03415245056e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:27:31.761346 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:31.761255 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b793b2e9-1e40-4e6c-96f5-a2c3b53fd982-kube-api-access-c9xf4" (OuterVolumeSpecName: "kube-api-access-c9xf4") pod "b793b2e9-1e40-4e6c-96f5-a2c3b53fd982" (UID: "b793b2e9-1e40-4e6c-96f5-a2c3b53fd982"). InnerVolumeSpecName "kube-api-access-c9xf4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 10:27:31.761346 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:31.761282 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db5968bb-1d95-4c73-8521-03415245056e-kube-api-access-pwb79" (OuterVolumeSpecName: "kube-api-access-pwb79") pod "db5968bb-1d95-4c73-8521-03415245056e" (UID: "db5968bb-1d95-4c73-8521-03415245056e"). InnerVolumeSpecName "kube-api-access-pwb79". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 10:27:31.761346 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:31.761281 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b793b2e9-1e40-4e6c-96f5-a2c3b53fd982-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "b793b2e9-1e40-4e6c-96f5-a2c3b53fd982" (UID: "b793b2e9-1e40-4e6c-96f5-a2c3b53fd982"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:27:31.761346 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:31.761293 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b793b2e9-1e40-4e6c-96f5-a2c3b53fd982-online-tls" (OuterVolumeSpecName: "online-tls") pod "b793b2e9-1e40-4e6c-96f5-a2c3b53fd982" (UID: "b793b2e9-1e40-4e6c-96f5-a2c3b53fd982"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:27:31.768455 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:31.768430 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db5968bb-1d95-4c73-8521-03415245056e-feast-data" (OuterVolumeSpecName: "feast-data") pod "db5968bb-1d95-4c73-8521-03415245056e" (UID: "db5968bb-1d95-4c73-8521-03415245056e"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 10:27:31.859383 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:31.859352 2568 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/db5968bb-1d95-4c73-8521-03415245056e-registry-tls\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:27:31.859383 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:31.859379 2568 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b793b2e9-1e40-4e6c-96f5-a2c3b53fd982-online-tls\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:27:31.859383 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:31.859387 2568 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b793b2e9-1e40-4e6c-96f5-a2c3b53fd982-feast-data\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:27:31.859558 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:31.859396 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9xf4\" (UniqueName: \"kubernetes.io/projected/b793b2e9-1e40-4e6c-96f5-a2c3b53fd982-kube-api-access-c9xf4\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:27:31.859558 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:31.859405 2568 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b793b2e9-1e40-4e6c-96f5-a2c3b53fd982-registry-tls\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:27:31.859558 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:31.859432 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwb79\" (UniqueName: \"kubernetes.io/projected/db5968bb-1d95-4c73-8521-03415245056e-kube-api-access-pwb79\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:27:31.859558 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:31.859440 2568 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/db5968bb-1d95-4c73-8521-03415245056e-feast-data\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:27:31.859558 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:31.859449 2568 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/db5968bb-1d95-4c73-8521-03415245056e-online-tls\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:27:32.381429 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:32.381385 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" Sep 08 10:27:32.381884 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:32.381435 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v" event={"ID":"db5968bb-1d95-4c73-8521-03415245056e","Type":"ContainerDied","Data":"18c82aa47a0c702f3e4fa11a985e7cdebba78539fc8b142cc9b5d8d586789fda"} Sep 08 10:27:32.381884 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:32.381502 2568 scope.go:117] "RemoveContainer" containerID="7c6cbb87a3a401067af5c09641c6594a3231765e17d62ca3f47efdbd54cf3df8" Sep 08 10:27:32.384063 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:32.384036 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" event={"ID":"b793b2e9-1e40-4e6c-96f5-a2c3b53fd982","Type":"ContainerDied","Data":"d575d24d0421fca8283b0a62e6accfa6279b5561f8486b7dc7a9a2d5b3dbd784"} Sep 08 10:27:32.384162 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:32.384090 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v" Sep 08 10:27:32.390030 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:32.390012 2568 scope.go:117] "RemoveContainer" containerID="8570e6cb907a1d0273287f7003003ae5d234f65c203a58426db013774264e10f" Sep 08 10:27:32.398539 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:32.398523 2568 scope.go:117] "RemoveContainer" containerID="b8e61a6a4cc1189d833b8c196dc20244d1dc41c8846f0b9e48e2634d4da129af" Sep 08 10:27:32.405479 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:32.405457 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v"] Sep 08 10:27:32.406934 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:32.406914 2568 scope.go:117] "RemoveContainer" containerID="965c2ee6b5648e75747e65c803be89ea3ceabef7869bae0ee64622f6139b2900" Sep 08 10:27:32.409490 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:32.409470 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-9d67f7657-5xr2v"] Sep 08 10:27:32.416997 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:32.416977 2568 scope.go:117] "RemoveContainer" containerID="9cef30b951df666da65b0aea144162c45fbdf7dc987affd6259be4c3a3c40808" Sep 08 10:27:32.421386 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:32.421369 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v"] Sep 08 10:27:32.424540 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:32.424522 2568 scope.go:117] "RemoveContainer" containerID="060187d913c69f180d96fd4c0282d287e8c96578e36351bb803032737305b37f" Sep 08 10:27:32.425286 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:32.425267 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7f487885db-dxs8v"] Sep 08 10:27:32.431712 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:32.431698 2568 scope.go:117] "RemoveContainer" containerID="dcd20c101e7ea0c274096d5e3731d93779a8545d0a07f491a2dda7c59785efad" Sep 08 10:27:32.438466 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:32.438448 2568 scope.go:117] "RemoveContainer" containerID="7ea3912ef8428acdb804417e1131a892386181f82924f8b3c00b9567091d839e" Sep 08 10:27:34.402612 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:34.402581 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b793b2e9-1e40-4e6c-96f5-a2c3b53fd982" path="/var/lib/kubelet/pods/b793b2e9-1e40-4e6c-96f5-a2c3b53fd982/volumes" Sep 08 10:27:34.403045 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:27:34.403033 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db5968bb-1d95-4c73-8521-03415245056e" path="/var/lib/kubelet/pods/db5968bb-1d95-4c73-8521-03415245056e/volumes" Sep 08 10:38:55.869720 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:55.869686 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kkc29/must-gather-z6kb9"] Sep 08 10:38:55.872085 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:55.869974 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83dcef83-236e-42af-bf8c-20d44fb5bfe0" containerName="redis" Sep 08 10:38:55.872085 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:55.869986 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="83dcef83-236e-42af-bf8c-20d44fb5bfe0" containerName="redis" Sep 08 10:38:55.872085 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:55.869998 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db5968bb-1d95-4c73-8521-03415245056e" containerName="online" Sep 08 10:38:55.872085 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:55.870007 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="db5968bb-1d95-4c73-8521-03415245056e" containerName="online" Sep 08 10:38:55.872085 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:55.870014 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b793b2e9-1e40-4e6c-96f5-a2c3b53fd982" containerName="registry" Sep 08 10:38:55.872085 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:55.870020 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b793b2e9-1e40-4e6c-96f5-a2c3b53fd982" containerName="registry" Sep 08 10:38:55.872085 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:55.870030 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bcb6d04-6fcf-467f-ae66-48a518ebb7b2" containerName="manager" Sep 08 10:38:55.872085 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:55.870036 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bcb6d04-6fcf-467f-ae66-48a518ebb7b2" containerName="manager" Sep 08 10:38:55.872085 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:55.870041 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b793b2e9-1e40-4e6c-96f5-a2c3b53fd982" containerName="feast-init" Sep 08 10:38:55.872085 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:55.870046 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b793b2e9-1e40-4e6c-96f5-a2c3b53fd982" containerName="feast-init" Sep 08 10:38:55.872085 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:55.870051 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b793b2e9-1e40-4e6c-96f5-a2c3b53fd982" containerName="online" Sep 08 10:38:55.872085 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:55.870056 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b793b2e9-1e40-4e6c-96f5-a2c3b53fd982" containerName="online" Sep 08 10:38:55.872085 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:55.870063 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b793b2e9-1e40-4e6c-96f5-a2c3b53fd982" containerName="feast-apply" Sep 08 10:38:55.872085 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:55.870067 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b793b2e9-1e40-4e6c-96f5-a2c3b53fd982" containerName="feast-apply" Sep 08 10:38:55.872085 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:55.870076 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db5968bb-1d95-4c73-8521-03415245056e" containerName="feast-apply" Sep 08 10:38:55.872085 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:55.870084 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="db5968bb-1d95-4c73-8521-03415245056e" containerName="feast-apply" Sep 08 10:38:55.872085 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:55.870096 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db5968bb-1d95-4c73-8521-03415245056e" containerName="registry" Sep 08 10:38:55.872085 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:55.870101 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="db5968bb-1d95-4c73-8521-03415245056e" containerName="registry" Sep 08 10:38:55.872085 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:55.870108 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b22ba9b6-0432-4e28-b72f-2176538d22ed" containerName="postgres" Sep 08 10:38:55.872085 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:55.870112 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b22ba9b6-0432-4e28-b72f-2176538d22ed" containerName="postgres" Sep 08 10:38:55.872085 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:55.870120 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db5968bb-1d95-4c73-8521-03415245056e" containerName="feast-init" Sep 08 10:38:55.872085 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:55.870125 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="db5968bb-1d95-4c73-8521-03415245056e" containerName="feast-init" Sep 08 10:38:55.872085 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:55.870174 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b793b2e9-1e40-4e6c-96f5-a2c3b53fd982" containerName="online" Sep 08 10:38:55.872085 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:55.870182 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bcb6d04-6fcf-467f-ae66-48a518ebb7b2" containerName="manager" Sep 08 10:38:55.872085 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:55.870190 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="db5968bb-1d95-4c73-8521-03415245056e" containerName="online" Sep 08 10:38:55.872085 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:55.870200 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b22ba9b6-0432-4e28-b72f-2176538d22ed" containerName="postgres" Sep 08 10:38:55.872085 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:55.870210 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b793b2e9-1e40-4e6c-96f5-a2c3b53fd982" containerName="registry" Sep 08 10:38:55.872085 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:55.870218 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="83dcef83-236e-42af-bf8c-20d44fb5bfe0" containerName="redis" Sep 08 10:38:55.872085 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:55.870226 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="db5968bb-1d95-4c73-8521-03415245056e" containerName="registry" Sep 08 10:38:55.873173 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:55.873158 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kkc29/must-gather-z6kb9" Sep 08 10:38:55.876333 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:55.876308 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kkc29\"/\"kube-root-ca.crt\"" Sep 08 10:38:55.876483 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:55.876399 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-kkc29\"/\"default-dockercfg-9v2h8\"" Sep 08 10:38:55.877327 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:55.877311 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kkc29\"/\"openshift-service-ca.crt\"" Sep 08 10:38:55.881359 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:55.881341 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kkc29/must-gather-z6kb9"] Sep 08 10:38:55.989792 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:55.989762 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zq2j\" (UniqueName: \"kubernetes.io/projected/e3f45dd5-f15a-4826-868c-b42447b149ba-kube-api-access-7zq2j\") pod \"must-gather-z6kb9\" (UID: \"e3f45dd5-f15a-4826-868c-b42447b149ba\") " pod="openshift-must-gather-kkc29/must-gather-z6kb9" Sep 08 10:38:55.989911 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:55.989817 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e3f45dd5-f15a-4826-868c-b42447b149ba-must-gather-output\") pod \"must-gather-z6kb9\" (UID: \"e3f45dd5-f15a-4826-868c-b42447b149ba\") " pod="openshift-must-gather-kkc29/must-gather-z6kb9" Sep 08 10:38:56.091130 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:56.091102 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e3f45dd5-f15a-4826-868c-b42447b149ba-must-gather-output\") pod \"must-gather-z6kb9\" (UID: \"e3f45dd5-f15a-4826-868c-b42447b149ba\") " pod="openshift-must-gather-kkc29/must-gather-z6kb9" Sep 08 10:38:56.091259 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:56.091149 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zq2j\" (UniqueName: \"kubernetes.io/projected/e3f45dd5-f15a-4826-868c-b42447b149ba-kube-api-access-7zq2j\") pod \"must-gather-z6kb9\" (UID: \"e3f45dd5-f15a-4826-868c-b42447b149ba\") " pod="openshift-must-gather-kkc29/must-gather-z6kb9" Sep 08 10:38:56.091455 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:56.091435 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e3f45dd5-f15a-4826-868c-b42447b149ba-must-gather-output\") pod \"must-gather-z6kb9\" (UID: \"e3f45dd5-f15a-4826-868c-b42447b149ba\") " pod="openshift-must-gather-kkc29/must-gather-z6kb9" Sep 08 10:38:56.100817 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:56.100796 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zq2j\" (UniqueName: \"kubernetes.io/projected/e3f45dd5-f15a-4826-868c-b42447b149ba-kube-api-access-7zq2j\") pod \"must-gather-z6kb9\" (UID: \"e3f45dd5-f15a-4826-868c-b42447b149ba\") " pod="openshift-must-gather-kkc29/must-gather-z6kb9" Sep 08 10:38:56.191572 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:56.191520 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kkc29/must-gather-z6kb9" Sep 08 10:38:56.470106 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:56.470082 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kkc29/must-gather-z6kb9"] Sep 08 10:38:56.472240 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:38:56.472218 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode3f45dd5_f15a_4826_868c_b42447b149ba.slice/crio-f44a060707e8c1c49011ef64f4880bf5af38436b4a8859ed9b5193dcfd071a49 WatchSource:0}: Error finding container f44a060707e8c1c49011ef64f4880bf5af38436b4a8859ed9b5193dcfd071a49: Status 404 returned error can't find the container with id f44a060707e8c1c49011ef64f4880bf5af38436b4a8859ed9b5193dcfd071a49 Sep 08 10:38:56.473939 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:56.473923 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 08 10:38:56.859866 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:38:56.859792 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kkc29/must-gather-z6kb9" event={"ID":"e3f45dd5-f15a-4826-868c-b42447b149ba","Type":"ContainerStarted","Data":"f44a060707e8c1c49011ef64f4880bf5af38436b4a8859ed9b5193dcfd071a49"} Sep 08 10:39:02.889295 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:02.889266 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kkc29/must-gather-z6kb9" event={"ID":"e3f45dd5-f15a-4826-868c-b42447b149ba","Type":"ContainerStarted","Data":"dfca5b448ab2c4c4fb2f41530023e069e6199fed76ae0cafe803d1b37788a314"} Sep 08 10:39:02.889691 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:02.889305 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kkc29/must-gather-z6kb9" event={"ID":"e3f45dd5-f15a-4826-868c-b42447b149ba","Type":"ContainerStarted","Data":"fec5c889997c97c66149524717eaeec717e2d9de4c625ffa6daaf5ce18b793b8"} Sep 08 10:39:02.909057 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:02.909012 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-kkc29/must-gather-z6kb9" podStartSLOduration=1.964352189 podStartE2EDuration="7.908995627s" totalImagesPullingTime="5.944643438s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:38:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:39:02.90771585 +0000 UTC m=+1926.951819919" watchObservedRunningTime="2026-09-08 10:39:02.908995627 +0000 UTC m=+1926.953099692" Sep 08 10:39:11.929812 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:11.929776 2568 generic.go:356] "Generic (PLEG): container finished" podID="e3f45dd5-f15a-4826-868c-b42447b149ba" containerID="fec5c889997c97c66149524717eaeec717e2d9de4c625ffa6daaf5ce18b793b8" exitCode=0 Sep 08 10:39:11.930160 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:11.929852 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kkc29/must-gather-z6kb9" event={"ID":"e3f45dd5-f15a-4826-868c-b42447b149ba","Type":"ContainerDied","Data":"fec5c889997c97c66149524717eaeec717e2d9de4c625ffa6daaf5ce18b793b8"} Sep 08 10:39:11.930160 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:11.930134 2568 scope.go:117] "RemoveContainer" containerID="fec5c889997c97c66149524717eaeec717e2d9de4c625ffa6daaf5ce18b793b8" Sep 08 10:39:12.870976 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:12.870943 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-kkc29_must-gather-z6kb9_e3f45dd5-f15a-4826-868c-b42447b149ba/gather/0.log" Sep 08 10:39:16.612201 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:16.612134 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-pv8mz_b608607f-8020-4d4e-9f43-b369b983a3b6/global-pull-secret-syncer/0.log" Sep 08 10:39:16.727893 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:16.727866 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-x4v9d_a6ea5044-04d6-44ae-a82e-35a4422fef08/konnectivity-agent/0.log" Sep 08 10:39:16.774598 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:16.774578 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-137.ec2.internal_a0020a1e2b212dc8803848e0c1e1357d/haproxy/0.log" Sep 08 10:39:18.201097 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:18.201064 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-kkc29/must-gather-z6kb9"] Sep 08 10:39:18.201576 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:18.201245 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-kkc29/must-gather-z6kb9" podUID="e3f45dd5-f15a-4826-868c-b42447b149ba" containerName="copy" containerID="cri-o://dfca5b448ab2c4c4fb2f41530023e069e6199fed76ae0cafe803d1b37788a314" gracePeriod=2 Sep 08 10:39:18.203228 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:18.203207 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-kkc29/must-gather-z6kb9"] Sep 08 10:39:18.203608 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:39:18.203576 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-z6kb9\" is forbidden: User \"system:node:ip-10-0-138-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-kkc29\": no relationship found between node 'ip-10-0-138-137.ec2.internal' and this object" podUID="e3f45dd5-f15a-4826-868c-b42447b149ba" pod="openshift-must-gather-kkc29/must-gather-z6kb9" Sep 08 10:39:18.426780 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:18.426760 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-kkc29_must-gather-z6kb9_e3f45dd5-f15a-4826-868c-b42447b149ba/copy/0.log" Sep 08 10:39:18.428343 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:18.428327 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kkc29/must-gather-z6kb9" Sep 08 10:39:18.563807 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:18.563745 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e3f45dd5-f15a-4826-868c-b42447b149ba-must-gather-output\") pod \"e3f45dd5-f15a-4826-868c-b42447b149ba\" (UID: \"e3f45dd5-f15a-4826-868c-b42447b149ba\") " Sep 08 10:39:18.563807 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:18.563792 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7zq2j\" (UniqueName: \"kubernetes.io/projected/e3f45dd5-f15a-4826-868c-b42447b149ba-kube-api-access-7zq2j\") pod \"e3f45dd5-f15a-4826-868c-b42447b149ba\" (UID: \"e3f45dd5-f15a-4826-868c-b42447b149ba\") " Sep 08 10:39:18.564062 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:18.564033 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3f45dd5-f15a-4826-868c-b42447b149ba-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "e3f45dd5-f15a-4826-868c-b42447b149ba" (UID: "e3f45dd5-f15a-4826-868c-b42447b149ba"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 10:39:18.565969 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:18.565948 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3f45dd5-f15a-4826-868c-b42447b149ba-kube-api-access-7zq2j" (OuterVolumeSpecName: "kube-api-access-7zq2j") pod "e3f45dd5-f15a-4826-868c-b42447b149ba" (UID: "e3f45dd5-f15a-4826-868c-b42447b149ba"). InnerVolumeSpecName "kube-api-access-7zq2j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 10:39:18.664610 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:18.664588 2568 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e3f45dd5-f15a-4826-868c-b42447b149ba-must-gather-output\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:39:18.664610 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:18.664608 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7zq2j\" (UniqueName: \"kubernetes.io/projected/e3f45dd5-f15a-4826-868c-b42447b149ba-kube-api-access-7zq2j\") on node \"ip-10-0-138-137.ec2.internal\" DevicePath \"\"" Sep 08 10:39:18.956240 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:18.956215 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-kkc29_must-gather-z6kb9_e3f45dd5-f15a-4826-868c-b42447b149ba/copy/0.log" Sep 08 10:39:18.956602 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:18.956580 2568 generic.go:356] "Generic (PLEG): container finished" podID="e3f45dd5-f15a-4826-868c-b42447b149ba" containerID="dfca5b448ab2c4c4fb2f41530023e069e6199fed76ae0cafe803d1b37788a314" exitCode=143 Sep 08 10:39:18.956685 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:18.956625 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kkc29/must-gather-z6kb9" Sep 08 10:39:18.956685 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:18.956650 2568 scope.go:117] "RemoveContainer" containerID="dfca5b448ab2c4c4fb2f41530023e069e6199fed76ae0cafe803d1b37788a314" Sep 08 10:39:18.964395 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:18.964379 2568 scope.go:117] "RemoveContainer" containerID="fec5c889997c97c66149524717eaeec717e2d9de4c625ffa6daaf5ce18b793b8" Sep 08 10:39:18.974686 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:18.974671 2568 scope.go:117] "RemoveContainer" containerID="dfca5b448ab2c4c4fb2f41530023e069e6199fed76ae0cafe803d1b37788a314" Sep 08 10:39:18.974969 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:39:18.974945 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfca5b448ab2c4c4fb2f41530023e069e6199fed76ae0cafe803d1b37788a314\": container with ID starting with dfca5b448ab2c4c4fb2f41530023e069e6199fed76ae0cafe803d1b37788a314 not found: ID does not exist" containerID="dfca5b448ab2c4c4fb2f41530023e069e6199fed76ae0cafe803d1b37788a314" Sep 08 10:39:18.975057 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:18.974973 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfca5b448ab2c4c4fb2f41530023e069e6199fed76ae0cafe803d1b37788a314"} err="failed to get container status \"dfca5b448ab2c4c4fb2f41530023e069e6199fed76ae0cafe803d1b37788a314\": rpc error: code = NotFound desc = could not find container \"dfca5b448ab2c4c4fb2f41530023e069e6199fed76ae0cafe803d1b37788a314\": container with ID starting with dfca5b448ab2c4c4fb2f41530023e069e6199fed76ae0cafe803d1b37788a314 not found: ID does not exist" Sep 08 10:39:18.975057 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:18.974991 2568 scope.go:117] "RemoveContainer" containerID="fec5c889997c97c66149524717eaeec717e2d9de4c625ffa6daaf5ce18b793b8" Sep 08 10:39:18.975225 ip-10-0-138-137 kubenswrapper[2568]: E0908 10:39:18.975211 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fec5c889997c97c66149524717eaeec717e2d9de4c625ffa6daaf5ce18b793b8\": container with ID starting with fec5c889997c97c66149524717eaeec717e2d9de4c625ffa6daaf5ce18b793b8 not found: ID does not exist" containerID="fec5c889997c97c66149524717eaeec717e2d9de4c625ffa6daaf5ce18b793b8" Sep 08 10:39:18.975265 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:18.975229 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fec5c889997c97c66149524717eaeec717e2d9de4c625ffa6daaf5ce18b793b8"} err="failed to get container status \"fec5c889997c97c66149524717eaeec717e2d9de4c625ffa6daaf5ce18b793b8\": rpc error: code = NotFound desc = could not find container \"fec5c889997c97c66149524717eaeec717e2d9de4c625ffa6daaf5ce18b793b8\": container with ID starting with fec5c889997c97c66149524717eaeec717e2d9de4c625ffa6daaf5ce18b793b8 not found: ID does not exist" Sep 08 10:39:20.403574 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:20.403548 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3f45dd5-f15a-4826-868c-b42447b149ba" path="/var/lib/kubelet/pods/e3f45dd5-f15a-4826-868c-b42447b149ba/volumes" Sep 08 10:39:20.442653 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:20.442633 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_afcc6060-56ad-4294-8d71-c535213cecd2/alertmanager/0.log" Sep 08 10:39:20.467516 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:20.467493 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_afcc6060-56ad-4294-8d71-c535213cecd2/config-reloader/0.log" Sep 08 10:39:20.492538 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:20.492513 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_afcc6060-56ad-4294-8d71-c535213cecd2/kube-rbac-proxy-web/0.log" Sep 08 10:39:20.517123 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:20.517102 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_afcc6060-56ad-4294-8d71-c535213cecd2/kube-rbac-proxy/0.log" Sep 08 10:39:20.537592 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:20.537571 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_afcc6060-56ad-4294-8d71-c535213cecd2/kube-rbac-proxy-metric/0.log" Sep 08 10:39:20.565113 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:20.565091 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_afcc6060-56ad-4294-8d71-c535213cecd2/prom-label-proxy/0.log" Sep 08 10:39:20.589105 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:20.589087 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_afcc6060-56ad-4294-8d71-c535213cecd2/init-config-reloader/0.log" Sep 08 10:39:21.025306 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:21.025279 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t589v_042a44d9-65fb-4808-bced-291b70524783/node-exporter/0.log" Sep 08 10:39:21.054069 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:21.054048 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t589v_042a44d9-65fb-4808-bced-291b70524783/kube-rbac-proxy/0.log" Sep 08 10:39:21.076849 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:21.076832 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t589v_042a44d9-65fb-4808-bced-291b70524783/init-textfile/0.log" Sep 08 10:39:21.112100 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:21.112077 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-zbkhn_64726051-51dc-4318-a02f-edbc884e3654/kube-rbac-proxy-main/0.log" Sep 08 10:39:21.139120 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:21.139099 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-zbkhn_64726051-51dc-4318-a02f-edbc884e3654/kube-rbac-proxy-self/0.log" Sep 08 10:39:21.162688 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:21.162664 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-zbkhn_64726051-51dc-4318-a02f-edbc884e3654/openshift-state-metrics/0.log" Sep 08 10:39:21.396057 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:21.396019 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-lvjmv_c58ff7e6-5d97-430e-95ae-7a86cfd47680/prometheus-operator/0.log" Sep 08 10:39:21.419845 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:21.419822 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-lvjmv_c58ff7e6-5d97-430e-95ae-7a86cfd47680/kube-rbac-proxy/0.log" Sep 08 10:39:21.451461 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:21.451435 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-f22s7_a7659944-4498-4a41-b2d3-e58f182c80e1/prometheus-operator-admission-webhook/0.log" Sep 08 10:39:23.751801 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:23.751766 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-zssmc/perf-node-gather-daemonset-892h7"] Sep 08 10:39:23.752285 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:23.752177 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3f45dd5-f15a-4826-868c-b42447b149ba" containerName="gather" Sep 08 10:39:23.752285 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:23.752195 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3f45dd5-f15a-4826-868c-b42447b149ba" containerName="gather" Sep 08 10:39:23.752285 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:23.752216 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3f45dd5-f15a-4826-868c-b42447b149ba" containerName="copy" Sep 08 10:39:23.752285 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:23.752224 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3f45dd5-f15a-4826-868c-b42447b149ba" containerName="copy" Sep 08 10:39:23.752534 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:23.752303 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3f45dd5-f15a-4826-868c-b42447b149ba" containerName="gather" Sep 08 10:39:23.752534 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:23.752315 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3f45dd5-f15a-4826-868c-b42447b149ba" containerName="copy" Sep 08 10:39:23.755611 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:23.755590 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zssmc/perf-node-gather-daemonset-892h7" Sep 08 10:39:23.758377 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:23.758355 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-zssmc\"/\"default-dockercfg-6zgz8\"" Sep 08 10:39:23.759243 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:23.759221 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zssmc\"/\"kube-root-ca.crt\"" Sep 08 10:39:23.759243 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:23.759240 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zssmc\"/\"openshift-service-ca.crt\"" Sep 08 10:39:23.764660 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:23.764640 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zssmc/perf-node-gather-daemonset-892h7"] Sep 08 10:39:23.890599 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:23.890570 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-bw9mh_b969204c-f958-48d9-9e4e-9827ed64496b/download-server/0.log" Sep 08 10:39:23.904248 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:23.904218 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d2cb389e-c6d0-42a7-b973-e9958452cbfd-proc\") pod \"perf-node-gather-daemonset-892h7\" (UID: \"d2cb389e-c6d0-42a7-b973-e9958452cbfd\") " pod="openshift-must-gather-zssmc/perf-node-gather-daemonset-892h7" Sep 08 10:39:23.904248 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:23.904251 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d2cb389e-c6d0-42a7-b973-e9958452cbfd-sys\") pod \"perf-node-gather-daemonset-892h7\" (UID: \"d2cb389e-c6d0-42a7-b973-e9958452cbfd\") " pod="openshift-must-gather-zssmc/perf-node-gather-daemonset-892h7" Sep 08 10:39:23.904490 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:23.904269 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d2cb389e-c6d0-42a7-b973-e9958452cbfd-lib-modules\") pod \"perf-node-gather-daemonset-892h7\" (UID: \"d2cb389e-c6d0-42a7-b973-e9958452cbfd\") " pod="openshift-must-gather-zssmc/perf-node-gather-daemonset-892h7" Sep 08 10:39:23.904490 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:23.904332 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d2cb389e-c6d0-42a7-b973-e9958452cbfd-podres\") pod \"perf-node-gather-daemonset-892h7\" (UID: \"d2cb389e-c6d0-42a7-b973-e9958452cbfd\") " pod="openshift-must-gather-zssmc/perf-node-gather-daemonset-892h7" Sep 08 10:39:23.904490 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:23.904452 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcsmm\" (UniqueName: \"kubernetes.io/projected/d2cb389e-c6d0-42a7-b973-e9958452cbfd-kube-api-access-qcsmm\") pod \"perf-node-gather-daemonset-892h7\" (UID: \"d2cb389e-c6d0-42a7-b973-e9958452cbfd\") " pod="openshift-must-gather-zssmc/perf-node-gather-daemonset-892h7" Sep 08 10:39:24.005086 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:24.004991 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcsmm\" (UniqueName: \"kubernetes.io/projected/d2cb389e-c6d0-42a7-b973-e9958452cbfd-kube-api-access-qcsmm\") pod \"perf-node-gather-daemonset-892h7\" (UID: \"d2cb389e-c6d0-42a7-b973-e9958452cbfd\") " pod="openshift-must-gather-zssmc/perf-node-gather-daemonset-892h7" Sep 08 10:39:24.005086 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:24.005035 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d2cb389e-c6d0-42a7-b973-e9958452cbfd-proc\") pod \"perf-node-gather-daemonset-892h7\" (UID: \"d2cb389e-c6d0-42a7-b973-e9958452cbfd\") " pod="openshift-must-gather-zssmc/perf-node-gather-daemonset-892h7" Sep 08 10:39:24.005086 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:24.005051 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d2cb389e-c6d0-42a7-b973-e9958452cbfd-sys\") pod \"perf-node-gather-daemonset-892h7\" (UID: \"d2cb389e-c6d0-42a7-b973-e9958452cbfd\") " pod="openshift-must-gather-zssmc/perf-node-gather-daemonset-892h7" Sep 08 10:39:24.005357 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:24.005123 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d2cb389e-c6d0-42a7-b973-e9958452cbfd-proc\") pod \"perf-node-gather-daemonset-892h7\" (UID: \"d2cb389e-c6d0-42a7-b973-e9958452cbfd\") " pod="openshift-must-gather-zssmc/perf-node-gather-daemonset-892h7" Sep 08 10:39:24.005357 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:24.005166 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d2cb389e-c6d0-42a7-b973-e9958452cbfd-lib-modules\") pod \"perf-node-gather-daemonset-892h7\" (UID: \"d2cb389e-c6d0-42a7-b973-e9958452cbfd\") " pod="openshift-must-gather-zssmc/perf-node-gather-daemonset-892h7" Sep 08 10:39:24.005357 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:24.005216 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d2cb389e-c6d0-42a7-b973-e9958452cbfd-sys\") pod \"perf-node-gather-daemonset-892h7\" (UID: \"d2cb389e-c6d0-42a7-b973-e9958452cbfd\") " pod="openshift-must-gather-zssmc/perf-node-gather-daemonset-892h7" Sep 08 10:39:24.005357 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:24.005223 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d2cb389e-c6d0-42a7-b973-e9958452cbfd-podres\") pod \"perf-node-gather-daemonset-892h7\" (UID: \"d2cb389e-c6d0-42a7-b973-e9958452cbfd\") " pod="openshift-must-gather-zssmc/perf-node-gather-daemonset-892h7" Sep 08 10:39:24.005357 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:24.005291 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d2cb389e-c6d0-42a7-b973-e9958452cbfd-lib-modules\") pod \"perf-node-gather-daemonset-892h7\" (UID: \"d2cb389e-c6d0-42a7-b973-e9958452cbfd\") " pod="openshift-must-gather-zssmc/perf-node-gather-daemonset-892h7" Sep 08 10:39:24.005357 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:24.005291 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d2cb389e-c6d0-42a7-b973-e9958452cbfd-podres\") pod \"perf-node-gather-daemonset-892h7\" (UID: \"d2cb389e-c6d0-42a7-b973-e9958452cbfd\") " pod="openshift-must-gather-zssmc/perf-node-gather-daemonset-892h7" Sep 08 10:39:24.016487 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:24.016457 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcsmm\" (UniqueName: \"kubernetes.io/projected/d2cb389e-c6d0-42a7-b973-e9958452cbfd-kube-api-access-qcsmm\") pod \"perf-node-gather-daemonset-892h7\" (UID: \"d2cb389e-c6d0-42a7-b973-e9958452cbfd\") " pod="openshift-must-gather-zssmc/perf-node-gather-daemonset-892h7" Sep 08 10:39:24.065623 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:24.065586 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zssmc/perf-node-gather-daemonset-892h7" Sep 08 10:39:24.337155 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:24.337074 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zssmc/perf-node-gather-daemonset-892h7"] Sep 08 10:39:24.340057 ip-10-0-138-137 kubenswrapper[2568]: W0908 10:39:24.340029 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd2cb389e_c6d0_42a7_b973_e9958452cbfd.slice/crio-ab16817eef00ccbda35876ead57ba69815938869de85c45eb95d80cf4c389750 WatchSource:0}: Error finding container ab16817eef00ccbda35876ead57ba69815938869de85c45eb95d80cf4c389750: Status 404 returned error can't find the container with id ab16817eef00ccbda35876ead57ba69815938869de85c45eb95d80cf4c389750 Sep 08 10:39:24.986317 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:24.986282 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zssmc/perf-node-gather-daemonset-892h7" event={"ID":"d2cb389e-c6d0-42a7-b973-e9958452cbfd","Type":"ContainerStarted","Data":"f0d22d634b0518cc9a60e72846758ce14c5fe7ff2f14ddfa06f95ba8b57ffdc4"} Sep 08 10:39:24.986724 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:24.986333 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zssmc/perf-node-gather-daemonset-892h7" event={"ID":"d2cb389e-c6d0-42a7-b973-e9958452cbfd","Type":"ContainerStarted","Data":"ab16817eef00ccbda35876ead57ba69815938869de85c45eb95d80cf4c389750"} Sep 08 10:39:24.986724 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:24.986369 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-zssmc/perf-node-gather-daemonset-892h7" Sep 08 10:39:25.005085 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:25.005037 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-zssmc/perf-node-gather-daemonset-892h7" podStartSLOduration=2.005022936 podStartE2EDuration="2.005022936s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 10:39:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:39:25.004282572 +0000 UTC m=+1949.048386662" watchObservedRunningTime="2026-09-08 10:39:25.005022936 +0000 UTC m=+1949.049127004" Sep 08 10:39:25.414550 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:25.414521 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dl46k_0045c129-e95a-4d80-a1b6-c972ee592d76/dns/0.log" Sep 08 10:39:25.435513 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:25.435478 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dl46k_0045c129-e95a-4d80-a1b6-c972ee592d76/kube-rbac-proxy/0.log" Sep 08 10:39:25.603711 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:25.603677 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vngnd_29e04309-88ba-49b2-aab0-4b68da8efd96/dns-node-resolver/0.log" Sep 08 10:39:26.056857 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:26.056829 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4gqq4_cfac6ed7-3115-4f50-9271-6a240f950ae5/node-ca/0.log" Sep 08 10:39:26.867216 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:26.867186 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-65cd7bbbb6-k5m7p_7d28e6ab-62c9-45a3-ae14-18623958e9ef/router/0.log" Sep 08 10:39:27.331329 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:27.331302 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rhxl4_7bfb2025-199a-4235-a283-7d555387fd2c/serve-healthcheck-canary/0.log" Sep 08 10:39:27.698301 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:27.698270 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-524nf_852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a/insights-operator/0.log" Sep 08 10:39:27.698831 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:27.698804 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-524nf_852f4bd7-e2d8-4ca8-b37a-b16df9ec9a1a/insights-operator/1.log" Sep 08 10:39:27.730753 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:27.730732 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-c5frh_c7957153-b146-457f-804c-55a93eb856a0/kube-rbac-proxy/0.log" Sep 08 10:39:27.758395 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:27.758368 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-c5frh_c7957153-b146-457f-804c-55a93eb856a0/exporter/0.log" Sep 08 10:39:27.782626 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:27.782608 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-c5frh_c7957153-b146-457f-804c-55a93eb856a0/extractor/0.log" Sep 08 10:39:31.000303 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:31.000271 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-zssmc/perf-node-gather-daemonset-892h7" Sep 08 10:39:33.939684 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:33.939655 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w5t97_9a781adb-4df3-440c-965a-1aa5e72f7271/kube-multus-additional-cni-plugins/0.log" Sep 08 10:39:33.959801 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:33.959778 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w5t97_9a781adb-4df3-440c-965a-1aa5e72f7271/egress-router-binary-copy/0.log" Sep 08 10:39:33.980148 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:33.980123 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w5t97_9a781adb-4df3-440c-965a-1aa5e72f7271/cni-plugins/0.log" Sep 08 10:39:34.000876 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:34.000859 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w5t97_9a781adb-4df3-440c-965a-1aa5e72f7271/bond-cni-plugin/0.log" Sep 08 10:39:34.025269 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:34.025246 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w5t97_9a781adb-4df3-440c-965a-1aa5e72f7271/routeoverride-cni/0.log" Sep 08 10:39:34.045997 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:34.045975 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w5t97_9a781adb-4df3-440c-965a-1aa5e72f7271/whereabouts-cni-bincopy/0.log" Sep 08 10:39:34.070855 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:34.070837 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w5t97_9a781adb-4df3-440c-965a-1aa5e72f7271/whereabouts-cni/0.log" Sep 08 10:39:34.111197 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:34.111175 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-p9x7d_a3abc8d2-5c6c-45f3-a8e9-b79d30b6b395/kube-multus/0.log" Sep 08 10:39:34.313936 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:34.313909 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mbs64_3b4e4397-1898-4201-9dcd-b5775ee1789c/network-metrics-daemon/0.log" Sep 08 10:39:34.340877 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:34.340853 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mbs64_3b4e4397-1898-4201-9dcd-b5775ee1789c/kube-rbac-proxy/0.log" Sep 08 10:39:35.403776 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:35.403748 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k296b_8ee82827-bbff-454b-b538-e76bf6fcf14d/ovn-controller/0.log" Sep 08 10:39:35.436842 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:35.436820 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k296b_8ee82827-bbff-454b-b538-e76bf6fcf14d/ovn-acl-logging/0.log" Sep 08 10:39:35.467343 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:35.467321 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k296b_8ee82827-bbff-454b-b538-e76bf6fcf14d/kube-rbac-proxy-node/0.log" Sep 08 10:39:35.497136 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:35.497111 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k296b_8ee82827-bbff-454b-b538-e76bf6fcf14d/kube-rbac-proxy-ovn-metrics/0.log" Sep 08 10:39:35.521047 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:35.521024 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k296b_8ee82827-bbff-454b-b538-e76bf6fcf14d/northd/0.log" Sep 08 10:39:35.546870 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:35.546852 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k296b_8ee82827-bbff-454b-b538-e76bf6fcf14d/nbdb/0.log" Sep 08 10:39:35.583571 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:35.583549 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k296b_8ee82827-bbff-454b-b538-e76bf6fcf14d/sbdb/0.log" Sep 08 10:39:35.642719 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:35.642695 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k296b_8ee82827-bbff-454b-b538-e76bf6fcf14d/ovnkube-controller/0.log" Sep 08 10:39:36.924229 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:36.924202 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-rx2jz_808f3bbd-c348-4c64-8ea7-e2e692594df8/network-check-target-container/0.log" Sep 08 10:39:37.849271 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:37.849241 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-7xfcp_2ea3bd6d-f1ab-4952-9091-c32bb4870a89/iptables-alerter/0.log" Sep 08 10:39:38.577377 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:38.577341 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-ntxft_ba8a6738-a445-4fdc-ad9e-e326d4ad1626/tuned/0.log" Sep 08 10:39:40.312585 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:40.312559 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-7cdcfcf76c-qs5nv_df6fbf45-0c71-4456-a95c-a5d5b24aa109/cluster-samples-operator/0.log" Sep 08 10:39:40.329855 ip-10-0-138-137 kubenswrapper[2568]: I0908 10:39:40.329831 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-7cdcfcf76c-qs5nv_df6fbf45-0c71-4456-a95c-a5d5b24aa109/cluster-samples-operator-watch/0.log"