Sep 09 02:08:34.267909 ip-10-0-128-218 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 09 02:08:34.267921 ip-10-0-128-218 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 09 02:08:34.267930 ip-10-0-128-218 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 09 02:08:34.268143 ip-10-0-128-218 systemd[1]: Failed to start Kubernetes Kubelet. Sep 09 02:08:44.447605 ip-10-0-128-218 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 09 02:08:44.447625 ip-10-0-128-218 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 31da9fbad5174c2783cf8c9f741d1724 -- Sep 09 02:10:40.510333 ip-10-0-128-218 systemd[1]: Starting Kubernetes Kubelet... Sep 09 02:10:40.893799 ip-10-0-128-218 kubenswrapper[2561]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 02:10:40.893799 ip-10-0-128-218 kubenswrapper[2561]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 09 02:10:40.893799 ip-10-0-128-218 kubenswrapper[2561]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 02:10:40.893799 ip-10-0-128-218 kubenswrapper[2561]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 09 02:10:40.893799 ip-10-0-128-218 kubenswrapper[2561]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 02:10:40.894844 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.894754 2561 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 09 02:10:40.898799 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898774 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 02:10:40.898799 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898793 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 02:10:40.898799 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898798 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 02:10:40.898799 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898802 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 02:10:40.898799 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898806 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 02:10:40.899133 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898811 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 02:10:40.899133 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898817 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 02:10:40.899133 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898821 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 02:10:40.899133 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898825 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 02:10:40.899133 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898829 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 02:10:40.899133 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898832 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 02:10:40.899133 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898836 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 02:10:40.899133 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898840 2561 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 02:10:40.899133 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898844 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 02:10:40.899133 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898848 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 02:10:40.899133 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898852 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 02:10:40.899133 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898857 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 02:10:40.899133 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898862 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 02:10:40.899133 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898872 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 02:10:40.899133 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898876 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 02:10:40.899133 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898880 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 02:10:40.899133 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898885 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 02:10:40.899133 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898889 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 02:10:40.899133 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898893 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 02:10:40.899922 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898897 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 02:10:40.899922 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898901 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 02:10:40.899922 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898904 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 02:10:40.899922 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898908 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 02:10:40.899922 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898912 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 02:10:40.899922 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898916 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 02:10:40.899922 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898920 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 02:10:40.899922 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898924 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 02:10:40.899922 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898928 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 02:10:40.899922 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898932 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 02:10:40.899922 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898936 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 02:10:40.899922 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898940 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 02:10:40.899922 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898944 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 02:10:40.899922 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898948 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 02:10:40.899922 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898954 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 02:10:40.899922 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898959 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 02:10:40.899922 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898964 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 02:10:40.899922 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898969 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 02:10:40.899922 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898974 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 02:10:40.899922 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898978 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 02:10:40.900785 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898985 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 02:10:40.900785 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898991 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 02:10:40.900785 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.898997 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 02:10:40.900785 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899001 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 02:10:40.900785 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899005 2561 feature_gate.go:328] unrecognized feature gate: Example Sep 09 02:10:40.900785 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899025 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 02:10:40.900785 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899030 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 02:10:40.900785 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899034 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 02:10:40.900785 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899039 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 02:10:40.900785 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899043 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 02:10:40.900785 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899047 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 02:10:40.900785 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899051 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 02:10:40.900785 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899054 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 02:10:40.900785 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899058 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 02:10:40.900785 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899065 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 02:10:40.900785 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899071 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 02:10:40.900785 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899076 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 02:10:40.900785 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899080 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 02:10:40.900785 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899085 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 02:10:40.901634 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899090 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 02:10:40.901634 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899094 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 02:10:40.901634 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899099 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 02:10:40.901634 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899103 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 02:10:40.901634 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899108 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 02:10:40.901634 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899113 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 02:10:40.901634 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899118 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 02:10:40.901634 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899127 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 02:10:40.901634 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899133 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 02:10:40.901634 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899137 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 02:10:40.901634 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899142 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 02:10:40.901634 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899146 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 02:10:40.901634 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899153 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 02:10:40.901634 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899157 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 02:10:40.901634 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899161 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 02:10:40.901634 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899166 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 02:10:40.901634 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899170 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 02:10:40.901634 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899174 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 02:10:40.901634 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899178 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 02:10:40.902460 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899182 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 02:10:40.902460 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899186 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 02:10:40.902460 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899190 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 02:10:40.902460 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899195 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 02:10:40.902460 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899198 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 02:10:40.902460 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899203 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 02:10:40.902460 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899207 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 02:10:40.902460 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899211 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 02:10:40.902460 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899215 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 02:10:40.902460 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899219 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 02:10:40.902460 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899223 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 02:10:40.902460 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899227 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 02:10:40.902460 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899232 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 02:10:40.902460 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899236 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 02:10:40.902460 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899241 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 02:10:40.902460 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899245 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 02:10:40.902460 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.899249 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 02:10:40.902460 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900396 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 02:10:40.902460 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900407 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 02:10:40.902460 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900412 2561 feature_gate.go:328] unrecognized feature gate: Example Sep 09 02:10:40.903303 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900419 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 02:10:40.903303 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900423 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 02:10:40.903303 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900428 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 02:10:40.903303 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900432 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 02:10:40.903303 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900439 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 02:10:40.903303 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900443 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 02:10:40.903303 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900447 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 02:10:40.903303 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900452 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 02:10:40.903303 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900456 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 02:10:40.903303 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900460 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 02:10:40.903303 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900465 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 02:10:40.903303 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900469 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 02:10:40.903303 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900473 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 02:10:40.903303 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900477 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 02:10:40.903303 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900481 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 02:10:40.903303 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900485 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 02:10:40.903303 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900489 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 02:10:40.903303 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900493 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 02:10:40.903303 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900497 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 02:10:40.903303 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900502 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 02:10:40.903807 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900506 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 02:10:40.903807 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900510 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 02:10:40.903807 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900514 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 02:10:40.903807 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900519 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 02:10:40.903807 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900523 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 02:10:40.903807 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900527 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 02:10:40.903807 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900531 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 02:10:40.903807 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900535 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 02:10:40.903807 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900539 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 02:10:40.903807 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900544 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 02:10:40.903807 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900548 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 02:10:40.903807 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900552 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 02:10:40.903807 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900557 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 02:10:40.903807 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900563 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 02:10:40.903807 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900567 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 02:10:40.903807 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900572 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 02:10:40.903807 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900576 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 02:10:40.903807 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900579 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 02:10:40.903807 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900583 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 02:10:40.904292 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900588 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 02:10:40.904292 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900596 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 02:10:40.904292 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900602 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 02:10:40.904292 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900607 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 02:10:40.904292 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900611 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 02:10:40.904292 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900616 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 02:10:40.904292 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900620 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 02:10:40.904292 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900625 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 02:10:40.904292 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900630 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 02:10:40.904292 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900634 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 02:10:40.904292 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900639 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 02:10:40.904292 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900643 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 02:10:40.904292 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900648 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 02:10:40.904292 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900652 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 02:10:40.904292 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900656 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 02:10:40.904292 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900660 2561 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 02:10:40.904292 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900665 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 02:10:40.904292 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900671 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 02:10:40.904292 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900676 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 02:10:40.904767 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900681 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 02:10:40.904767 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900686 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 02:10:40.904767 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900690 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 02:10:40.904767 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900694 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 02:10:40.904767 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900699 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 02:10:40.904767 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900703 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 02:10:40.904767 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900708 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 02:10:40.904767 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900713 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 02:10:40.904767 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900719 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 02:10:40.904767 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900725 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 02:10:40.904767 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900729 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 02:10:40.904767 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900734 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 02:10:40.904767 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900738 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 02:10:40.904767 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900742 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 02:10:40.904767 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900746 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 02:10:40.904767 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900752 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 02:10:40.904767 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900757 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 02:10:40.904767 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900761 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 02:10:40.904767 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900765 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 02:10:40.904767 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900769 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 02:10:40.905296 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900773 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 02:10:40.905296 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900777 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 02:10:40.905296 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900781 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 02:10:40.905296 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900785 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 02:10:40.905296 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900789 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 02:10:40.905296 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900793 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 02:10:40.905296 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900797 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 02:10:40.905296 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900803 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 02:10:40.905296 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900808 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 02:10:40.905296 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900812 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 02:10:40.905296 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900817 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 02:10:40.905296 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900821 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 02:10:40.905296 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900825 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 02:10:40.905296 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900829 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 02:10:40.905296 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900833 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 02:10:40.905296 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900837 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 02:10:40.905296 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900841 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 02:10:40.905296 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.900846 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 02:10:40.905296 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901478 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 02:10:40.905834 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901487 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 02:10:40.905834 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901496 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 02:10:40.905834 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901501 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 02:10:40.905834 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901505 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 02:10:40.905834 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901509 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 02:10:40.905834 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901514 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 02:10:40.905834 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901518 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 02:10:40.905834 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901522 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 02:10:40.905834 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901527 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 02:10:40.905834 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901532 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 02:10:40.905834 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901537 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 02:10:40.905834 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901540 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 02:10:40.905834 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901544 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 02:10:40.905834 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901549 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 02:10:40.905834 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901552 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 02:10:40.905834 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901557 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 02:10:40.905834 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901560 2561 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 02:10:40.905834 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901565 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 02:10:40.905834 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901569 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 02:10:40.905834 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901573 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 02:10:40.906402 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901582 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 02:10:40.906402 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901587 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 02:10:40.906402 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901591 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 02:10:40.906402 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901596 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 02:10:40.906402 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901600 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 02:10:40.906402 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901605 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 02:10:40.906402 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901609 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 02:10:40.906402 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901613 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 02:10:40.906402 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901618 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 02:10:40.906402 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901622 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 02:10:40.906402 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901627 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 02:10:40.906402 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901631 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 02:10:40.906402 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901636 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 02:10:40.906402 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901640 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 02:10:40.906402 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901645 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 02:10:40.906402 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901649 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 02:10:40.906402 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901654 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 02:10:40.906402 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901659 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 02:10:40.906402 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901663 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 02:10:40.906402 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901667 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 02:10:40.906948 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901671 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 02:10:40.906948 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901676 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 02:10:40.906948 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901680 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 02:10:40.906948 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901685 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 02:10:40.906948 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901690 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 02:10:40.906948 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901697 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 02:10:40.906948 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901701 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 02:10:40.906948 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901712 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 02:10:40.906948 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901717 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 02:10:40.906948 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901721 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 02:10:40.906948 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901725 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 02:10:40.906948 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901730 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 02:10:40.906948 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901734 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 02:10:40.906948 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901738 2561 feature_gate.go:328] unrecognized feature gate: Example Sep 09 02:10:40.906948 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901742 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 02:10:40.906948 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901746 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 02:10:40.906948 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901750 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 02:10:40.906948 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901754 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 02:10:40.906948 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901758 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 02:10:40.906948 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901763 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 02:10:40.907544 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901767 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 02:10:40.907544 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901771 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 02:10:40.907544 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901776 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 02:10:40.907544 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901781 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 02:10:40.907544 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901785 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 02:10:40.907544 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901789 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 02:10:40.907544 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901793 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 02:10:40.907544 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901799 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 02:10:40.907544 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901803 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 02:10:40.907544 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901807 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 02:10:40.907544 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901811 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 02:10:40.907544 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901815 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 02:10:40.907544 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901819 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 02:10:40.907544 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901823 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 02:10:40.907544 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901828 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 02:10:40.907544 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901832 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 02:10:40.907544 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901836 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 02:10:40.907544 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901840 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 02:10:40.907544 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901844 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 02:10:40.907544 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901848 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 02:10:40.908103 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901853 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 02:10:40.908103 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901858 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 02:10:40.908103 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901862 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 02:10:40.908103 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901866 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 02:10:40.908103 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901871 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 02:10:40.908103 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901875 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 02:10:40.908103 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901879 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 02:10:40.908103 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901886 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 02:10:40.908103 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901891 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 02:10:40.908103 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901897 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 02:10:40.908103 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901903 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 02:10:40.908103 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901907 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 02:10:40.908103 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901913 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 02:10:40.908103 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901917 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 02:10:40.908103 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901922 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 02:10:40.908103 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901928 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 02:10:40.908103 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901933 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 02:10:40.908103 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.901938 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 02:10:40.908103 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902063 2561 flags.go:64] FLAG: --address="0.0.0.0" Sep 09 02:10:40.908630 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902119 2561 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 09 02:10:40.908630 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902129 2561 flags.go:64] FLAG: --anonymous-auth="true" Sep 09 02:10:40.908630 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902136 2561 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 09 02:10:40.908630 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902143 2561 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 09 02:10:40.908630 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902149 2561 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 09 02:10:40.908630 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902155 2561 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 09 02:10:40.908630 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902162 2561 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 09 02:10:40.908630 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902167 2561 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 09 02:10:40.908630 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902172 2561 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 09 02:10:40.908630 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902177 2561 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 09 02:10:40.908630 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902183 2561 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 09 02:10:40.908630 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902188 2561 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 09 02:10:40.908630 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902193 2561 flags.go:64] FLAG: --cgroup-root="" Sep 09 02:10:40.908630 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902198 2561 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 09 02:10:40.908630 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902203 2561 flags.go:64] FLAG: --client-ca-file="" Sep 09 02:10:40.908630 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902208 2561 flags.go:64] FLAG: --cloud-provider="external" Sep 09 02:10:40.908630 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902213 2561 flags.go:64] FLAG: --cluster-dns="[]" Sep 09 02:10:40.908630 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902220 2561 flags.go:64] FLAG: --cluster-domain="" Sep 09 02:10:40.908630 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902226 2561 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 09 02:10:40.908630 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902231 2561 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 09 02:10:40.908630 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902236 2561 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 09 02:10:40.908630 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902241 2561 flags.go:64] FLAG: --container-log-max-files="5" Sep 09 02:10:40.908630 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902247 2561 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 09 02:10:40.908630 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902252 2561 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 09 02:10:40.909302 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902257 2561 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 09 02:10:40.909302 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902263 2561 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 09 02:10:40.909302 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902267 2561 flags.go:64] FLAG: --contention-profiling="false" Sep 09 02:10:40.909302 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902276 2561 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 09 02:10:40.909302 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902281 2561 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 09 02:10:40.909302 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902286 2561 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 09 02:10:40.909302 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902290 2561 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 09 02:10:40.909302 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902297 2561 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 09 02:10:40.909302 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902303 2561 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 09 02:10:40.909302 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902307 2561 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 09 02:10:40.909302 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902313 2561 flags.go:64] FLAG: --enable-load-reader="false" Sep 09 02:10:40.909302 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902318 2561 flags.go:64] FLAG: --enable-server="true" Sep 09 02:10:40.909302 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902322 2561 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 09 02:10:40.909302 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902330 2561 flags.go:64] FLAG: --event-burst="100" Sep 09 02:10:40.909302 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902334 2561 flags.go:64] FLAG: --event-qps="50" Sep 09 02:10:40.909302 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902339 2561 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 09 02:10:40.909302 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902344 2561 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 09 02:10:40.909302 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902349 2561 flags.go:64] FLAG: --eviction-hard="" Sep 09 02:10:40.909302 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902354 2561 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 09 02:10:40.909302 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902359 2561 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 09 02:10:40.909302 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902364 2561 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 09 02:10:40.909302 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902369 2561 flags.go:64] FLAG: --eviction-soft="" Sep 09 02:10:40.909302 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902373 2561 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 09 02:10:40.909302 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902378 2561 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 09 02:10:40.909302 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902383 2561 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 09 02:10:40.909993 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902387 2561 flags.go:64] FLAG: --experimental-mounter-path="" Sep 09 02:10:40.909993 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902392 2561 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 09 02:10:40.909993 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902397 2561 flags.go:64] FLAG: --fail-swap-on="true" Sep 09 02:10:40.909993 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902402 2561 flags.go:64] FLAG: --feature-gates="" Sep 09 02:10:40.909993 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902408 2561 flags.go:64] FLAG: --file-check-frequency="20s" Sep 09 02:10:40.909993 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902412 2561 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 09 02:10:40.909993 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902417 2561 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 09 02:10:40.909993 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902422 2561 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 09 02:10:40.909993 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902427 2561 flags.go:64] FLAG: --healthz-port="10248" Sep 09 02:10:40.909993 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902432 2561 flags.go:64] FLAG: --help="false" Sep 09 02:10:40.909993 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902440 2561 flags.go:64] FLAG: --hostname-override="ip-10-0-128-218.ec2.internal" Sep 09 02:10:40.909993 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902445 2561 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 09 02:10:40.909993 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902450 2561 flags.go:64] FLAG: --http-check-frequency="20s" Sep 09 02:10:40.909993 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902455 2561 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 09 02:10:40.909993 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902460 2561 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 09 02:10:40.909993 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902466 2561 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 09 02:10:40.909993 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902470 2561 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 09 02:10:40.909993 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902475 2561 flags.go:64] FLAG: --image-service-endpoint="" Sep 09 02:10:40.909993 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902480 2561 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 09 02:10:40.909993 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902485 2561 flags.go:64] FLAG: --kube-api-burst="100" Sep 09 02:10:40.909993 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902489 2561 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 09 02:10:40.909993 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902494 2561 flags.go:64] FLAG: --kube-api-qps="50" Sep 09 02:10:40.909993 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902499 2561 flags.go:64] FLAG: --kube-reserved="" Sep 09 02:10:40.909993 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902504 2561 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 09 02:10:40.910617 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902509 2561 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 09 02:10:40.910617 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902514 2561 flags.go:64] FLAG: --kubelet-cgroups="" Sep 09 02:10:40.910617 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902519 2561 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 09 02:10:40.910617 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902523 2561 flags.go:64] FLAG: --lock-file="" Sep 09 02:10:40.910617 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902528 2561 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 09 02:10:40.910617 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902532 2561 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 09 02:10:40.910617 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902537 2561 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 09 02:10:40.910617 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902546 2561 flags.go:64] FLAG: --log-json-split-stream="false" Sep 09 02:10:40.910617 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902554 2561 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 09 02:10:40.910617 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902559 2561 flags.go:64] FLAG: --log-text-split-stream="false" Sep 09 02:10:40.910617 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902564 2561 flags.go:64] FLAG: --logging-format="text" Sep 09 02:10:40.910617 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902569 2561 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 09 02:10:40.910617 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902574 2561 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 09 02:10:40.910617 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902579 2561 flags.go:64] FLAG: --manifest-url="" Sep 09 02:10:40.910617 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902583 2561 flags.go:64] FLAG: --manifest-url-header="" Sep 09 02:10:40.910617 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902591 2561 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 09 02:10:40.910617 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902596 2561 flags.go:64] FLAG: --max-open-files="1000000" Sep 09 02:10:40.910617 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902602 2561 flags.go:64] FLAG: --max-pods="110" Sep 09 02:10:40.910617 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902609 2561 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 09 02:10:40.910617 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902613 2561 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 09 02:10:40.910617 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902618 2561 flags.go:64] FLAG: --memory-manager-policy="None" Sep 09 02:10:40.910617 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902623 2561 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 09 02:10:40.910617 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902628 2561 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 09 02:10:40.910617 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902633 2561 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 09 02:10:40.910617 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902637 2561 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 09 02:10:40.911708 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902649 2561 flags.go:64] FLAG: --node-status-max-images="50" Sep 09 02:10:40.911708 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902653 2561 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 09 02:10:40.911708 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902659 2561 flags.go:64] FLAG: --oom-score-adj="-999" Sep 09 02:10:40.911708 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902664 2561 flags.go:64] FLAG: --pod-cidr="" Sep 09 02:10:40.911708 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902669 2561 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 09 02:10:40.911708 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902677 2561 flags.go:64] FLAG: --pod-manifest-path="" Sep 09 02:10:40.911708 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902682 2561 flags.go:64] FLAG: --pod-max-pids="-1" Sep 09 02:10:40.911708 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902687 2561 flags.go:64] FLAG: --pods-per-core="0" Sep 09 02:10:40.911708 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902692 2561 flags.go:64] FLAG: --port="10250" Sep 09 02:10:40.911708 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902696 2561 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 09 02:10:40.911708 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902701 2561 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0fc2f75a982aded28" Sep 09 02:10:40.911708 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902706 2561 flags.go:64] FLAG: --qos-reserved="" Sep 09 02:10:40.911708 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902711 2561 flags.go:64] FLAG: --read-only-port="10255" Sep 09 02:10:40.911708 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902716 2561 flags.go:64] FLAG: --register-node="true" Sep 09 02:10:40.911708 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902720 2561 flags.go:64] FLAG: --register-with-taints="" Sep 09 02:10:40.911708 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902726 2561 flags.go:64] FLAG: --registry-burst="10" Sep 09 02:10:40.911708 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902733 2561 flags.go:64] FLAG: --registry-qps="5" Sep 09 02:10:40.911708 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902737 2561 flags.go:64] FLAG: --reserved-cpus="" Sep 09 02:10:40.911708 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902742 2561 flags.go:64] FLAG: --reserved-memory="" Sep 09 02:10:40.911708 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902749 2561 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 09 02:10:40.911708 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902754 2561 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 09 02:10:40.911708 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902759 2561 flags.go:64] FLAG: --rotate-certificates="false" Sep 09 02:10:40.911708 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902763 2561 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 09 02:10:40.911708 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902768 2561 flags.go:64] FLAG: --runonce="false" Sep 09 02:10:40.911708 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902772 2561 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 09 02:10:40.912369 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902780 2561 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 09 02:10:40.912369 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902785 2561 flags.go:64] FLAG: --seccomp-default="false" Sep 09 02:10:40.912369 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902790 2561 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 09 02:10:40.912369 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902794 2561 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 09 02:10:40.912369 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902799 2561 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 09 02:10:40.912369 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902804 2561 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 09 02:10:40.912369 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902809 2561 flags.go:64] FLAG: --storage-driver-password="root" Sep 09 02:10:40.912369 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902814 2561 flags.go:64] FLAG: --storage-driver-secure="false" Sep 09 02:10:40.912369 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902819 2561 flags.go:64] FLAG: --storage-driver-table="stats" Sep 09 02:10:40.912369 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902824 2561 flags.go:64] FLAG: --storage-driver-user="root" Sep 09 02:10:40.912369 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902829 2561 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 09 02:10:40.912369 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902834 2561 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 09 02:10:40.912369 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902839 2561 flags.go:64] FLAG: --system-cgroups="" Sep 09 02:10:40.912369 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902844 2561 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 09 02:10:40.912369 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902852 2561 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 09 02:10:40.912369 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902857 2561 flags.go:64] FLAG: --tls-cert-file="" Sep 09 02:10:40.912369 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902861 2561 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 09 02:10:40.912369 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902868 2561 flags.go:64] FLAG: --tls-min-version="" Sep 09 02:10:40.912369 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902873 2561 flags.go:64] FLAG: --tls-private-key-file="" Sep 09 02:10:40.912369 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902878 2561 flags.go:64] FLAG: --topology-manager-policy="none" Sep 09 02:10:40.912369 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902882 2561 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 09 02:10:40.912369 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902887 2561 flags.go:64] FLAG: --topology-manager-scope="container" Sep 09 02:10:40.912369 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902892 2561 flags.go:64] FLAG: --v="2" Sep 09 02:10:40.912369 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902900 2561 flags.go:64] FLAG: --version="false" Sep 09 02:10:40.912369 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902906 2561 flags.go:64] FLAG: --vmodule="" Sep 09 02:10:40.913003 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902912 2561 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 09 02:10:40.913003 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.902917 2561 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 09 02:10:40.913003 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903093 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 02:10:40.913003 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903099 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 02:10:40.913003 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903102 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 02:10:40.913003 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903105 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 02:10:40.913003 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903107 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 02:10:40.913003 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903112 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 02:10:40.913003 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903114 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 02:10:40.913003 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903117 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 02:10:40.913003 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903120 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 02:10:40.913003 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903122 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 02:10:40.913003 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903125 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 02:10:40.913003 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903128 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 02:10:40.913003 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903130 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 02:10:40.913003 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903133 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 02:10:40.913003 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903136 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 02:10:40.913003 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903138 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 02:10:40.913003 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903142 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 02:10:40.913003 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903145 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 02:10:40.913522 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903148 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 02:10:40.913522 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903150 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 02:10:40.913522 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903152 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 02:10:40.913522 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903155 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 02:10:40.913522 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903158 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 02:10:40.913522 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903160 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 02:10:40.913522 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903163 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 02:10:40.913522 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903165 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 02:10:40.913522 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903168 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 02:10:40.913522 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903170 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 02:10:40.913522 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903174 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 02:10:40.913522 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903177 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 02:10:40.913522 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903179 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 02:10:40.913522 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903182 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 02:10:40.913522 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903185 2561 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 02:10:40.913522 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903187 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 02:10:40.913522 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903190 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 02:10:40.913522 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903192 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 02:10:40.913522 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903195 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 02:10:40.913522 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903199 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 02:10:40.914040 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903202 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 02:10:40.914040 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903204 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 02:10:40.914040 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903207 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 02:10:40.914040 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903210 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 02:10:40.914040 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903212 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 02:10:40.914040 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903215 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 02:10:40.914040 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903217 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 02:10:40.914040 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903220 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 02:10:40.914040 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903223 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 02:10:40.914040 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903225 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 02:10:40.914040 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903228 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 02:10:40.914040 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903231 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 02:10:40.914040 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903233 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 02:10:40.914040 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903235 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 02:10:40.914040 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903238 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 02:10:40.914040 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903240 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 02:10:40.914040 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903243 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 02:10:40.914040 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903246 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 02:10:40.914040 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903248 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 02:10:40.914520 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903251 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 02:10:40.914520 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903253 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 02:10:40.914520 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903257 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 02:10:40.914520 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903262 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 02:10:40.914520 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903265 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 02:10:40.914520 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903268 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 02:10:40.914520 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903270 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 02:10:40.914520 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903273 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 02:10:40.914520 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903276 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 02:10:40.914520 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903278 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 02:10:40.914520 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903281 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 02:10:40.914520 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903283 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 02:10:40.914520 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903288 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 02:10:40.914520 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903291 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 02:10:40.914520 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903293 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 02:10:40.914520 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903296 2561 feature_gate.go:328] unrecognized feature gate: Example Sep 09 02:10:40.914520 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903300 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 02:10:40.914520 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903304 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 02:10:40.914520 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903307 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 02:10:40.914982 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903309 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 02:10:40.914982 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903312 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 02:10:40.914982 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903316 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 02:10:40.914982 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903320 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 02:10:40.914982 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903323 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 02:10:40.914982 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903326 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 02:10:40.914982 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903329 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 02:10:40.914982 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903332 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 02:10:40.914982 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903334 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 02:10:40.914982 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903337 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 02:10:40.914982 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903339 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 02:10:40.914982 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903342 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 02:10:40.914982 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903344 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 02:10:40.914982 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903347 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 02:10:40.914982 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903349 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 02:10:40.914982 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903352 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 02:10:40.914982 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903355 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 02:10:40.914982 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903358 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 02:10:40.914982 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903360 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 02:10:40.915554 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903363 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 02:10:40.915554 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903366 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 02:10:40.915554 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903368 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 02:10:40.915554 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.903371 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 02:10:40.915554 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.904332 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 02:10:40.915554 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.913555 2561 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 09 02:10:40.915554 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.913571 2561 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 09 02:10:40.915554 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.914324 2561 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 09 02:10:40.915554 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.914336 2561 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 09 02:10:40.915554 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914394 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 02:10:40.915554 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914399 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 02:10:40.915554 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914403 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 02:10:40.915554 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914406 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 02:10:40.915554 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914409 2561 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 02:10:40.915554 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914412 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 02:10:40.915938 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914415 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 02:10:40.915938 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914418 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 02:10:40.915938 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914420 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 02:10:40.915938 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914423 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 02:10:40.915938 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914426 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 02:10:40.915938 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914429 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 02:10:40.915938 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914432 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 02:10:40.915938 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914434 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 02:10:40.915938 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914437 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 02:10:40.915938 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914440 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 02:10:40.915938 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914443 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 02:10:40.915938 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914445 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 02:10:40.915938 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914448 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 02:10:40.915938 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914451 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 02:10:40.915938 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914453 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 02:10:40.915938 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914456 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 02:10:40.915938 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914458 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 02:10:40.915938 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914461 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 02:10:40.915938 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914463 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 02:10:40.915938 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914466 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 02:10:40.916448 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914469 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 02:10:40.916448 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914472 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 02:10:40.916448 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914475 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 02:10:40.916448 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914478 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 02:10:40.916448 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914481 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 02:10:40.916448 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914484 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 02:10:40.916448 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914487 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 02:10:40.916448 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914489 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 02:10:40.916448 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914492 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 02:10:40.916448 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914495 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 02:10:40.916448 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914497 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 02:10:40.916448 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914500 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 02:10:40.916448 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914502 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 02:10:40.916448 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914505 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 02:10:40.916448 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914508 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 02:10:40.916448 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914510 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 02:10:40.916448 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914513 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 02:10:40.916448 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914515 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 02:10:40.916448 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914518 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 02:10:40.916448 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914520 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 02:10:40.916937 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914525 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 02:10:40.916937 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914529 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 02:10:40.916937 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914532 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 02:10:40.916937 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914535 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 02:10:40.916937 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914538 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 02:10:40.916937 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914541 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 02:10:40.916937 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914544 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 02:10:40.916937 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914565 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 02:10:40.916937 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914570 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 02:10:40.916937 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914573 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 02:10:40.916937 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914576 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 02:10:40.916937 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914578 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 02:10:40.916937 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914582 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 02:10:40.916937 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914586 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 02:10:40.916937 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914589 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 02:10:40.916937 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914592 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 02:10:40.916937 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914596 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 02:10:40.916937 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914599 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 02:10:40.916937 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914601 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 02:10:40.917419 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914604 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 02:10:40.917419 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914607 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 02:10:40.917419 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914609 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 02:10:40.917419 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914612 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 02:10:40.917419 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914615 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 02:10:40.917419 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914617 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 02:10:40.917419 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914620 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 02:10:40.917419 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914623 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 02:10:40.917419 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914625 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 02:10:40.917419 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914628 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 02:10:40.917419 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914631 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 02:10:40.917419 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914633 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 02:10:40.917419 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914636 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 02:10:40.917419 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914639 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 02:10:40.917419 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914644 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 02:10:40.917419 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914647 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 02:10:40.917419 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914649 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 02:10:40.917419 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914652 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 02:10:40.917419 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914654 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 02:10:40.917890 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914657 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 02:10:40.917890 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914660 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 02:10:40.917890 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914662 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 02:10:40.917890 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914665 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 02:10:40.917890 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914667 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 02:10:40.917890 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914670 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 02:10:40.917890 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914672 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 02:10:40.917890 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914675 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 02:10:40.917890 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914677 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 02:10:40.917890 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914680 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 02:10:40.917890 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914683 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 02:10:40.917890 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914686 2561 feature_gate.go:328] unrecognized feature gate: Example Sep 09 02:10:40.917890 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914689 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 02:10:40.917890 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914692 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 02:10:40.917890 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914694 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 02:10:40.918279 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.914699 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 02:10:40.918279 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914823 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 02:10:40.918279 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914828 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 02:10:40.918279 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914831 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 02:10:40.918279 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914834 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 02:10:40.918279 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914837 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 02:10:40.918279 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914839 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 02:10:40.918279 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914842 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 02:10:40.918279 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914845 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 02:10:40.918279 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914848 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 02:10:40.918279 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914850 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 02:10:40.918279 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914853 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 02:10:40.918279 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914855 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 02:10:40.918279 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914864 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 02:10:40.918279 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914867 2561 feature_gate.go:328] unrecognized feature gate: Example Sep 09 02:10:40.918647 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914869 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 02:10:40.918647 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914873 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 02:10:40.918647 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914876 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 02:10:40.918647 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914879 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 02:10:40.918647 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914881 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 02:10:40.918647 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914884 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 02:10:40.918647 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914887 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 02:10:40.918647 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914889 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 02:10:40.918647 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914892 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 02:10:40.918647 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914895 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 02:10:40.918647 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914898 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 02:10:40.918647 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914900 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 02:10:40.918647 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914905 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 02:10:40.918647 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914909 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 02:10:40.918647 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914912 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 02:10:40.918647 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914915 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 02:10:40.918647 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914918 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 02:10:40.918647 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914921 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 02:10:40.918647 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914923 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 02:10:40.919125 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914926 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 02:10:40.919125 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914929 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 02:10:40.919125 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914932 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 02:10:40.919125 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914934 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 02:10:40.919125 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914936 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 02:10:40.919125 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914939 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 02:10:40.919125 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914942 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 02:10:40.919125 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914944 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 02:10:40.919125 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914947 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 02:10:40.919125 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914949 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 02:10:40.919125 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914952 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 02:10:40.919125 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914954 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 02:10:40.919125 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914957 2561 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 02:10:40.919125 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914960 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 02:10:40.919125 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914963 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 02:10:40.919125 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914966 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 02:10:40.919125 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914968 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 02:10:40.919125 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914971 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 02:10:40.919125 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914974 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 02:10:40.919125 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914976 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 02:10:40.919618 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914979 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 02:10:40.919618 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914982 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 02:10:40.919618 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914984 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 02:10:40.919618 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914987 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 02:10:40.919618 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914990 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 02:10:40.919618 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914993 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 02:10:40.919618 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914995 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 02:10:40.919618 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.914998 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 02:10:40.919618 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.915000 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 02:10:40.919618 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.915003 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 02:10:40.919618 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.915005 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 02:10:40.919618 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.915008 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 02:10:40.919618 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.915025 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 02:10:40.919618 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.915030 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 02:10:40.919618 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.915034 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 02:10:40.919618 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.915037 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 02:10:40.919618 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.915040 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 02:10:40.919618 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.915042 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 02:10:40.919618 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.915045 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 02:10:40.919618 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.915048 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 02:10:40.920201 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.915050 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 02:10:40.920201 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.915054 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 02:10:40.920201 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.915059 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 02:10:40.920201 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.915062 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 02:10:40.920201 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.915064 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 02:10:40.920201 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.915068 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 02:10:40.920201 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.915071 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 02:10:40.920201 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.915074 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 02:10:40.920201 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.915077 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 02:10:40.920201 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.915079 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 02:10:40.920201 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.915082 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 02:10:40.920201 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.915085 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 02:10:40.920201 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.915087 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 02:10:40.920201 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.915090 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 02:10:40.920201 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.915093 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 02:10:40.920201 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.915095 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 02:10:40.920201 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.915098 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 02:10:40.920201 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.915102 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 02:10:40.920201 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.915105 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 02:10:40.920760 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.915108 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 02:10:40.920760 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.915111 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 02:10:40.920760 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.915113 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 02:10:40.920760 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.915116 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 02:10:40.920760 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.915118 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 02:10:40.920760 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.915121 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 02:10:40.920760 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:40.915124 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 02:10:40.920760 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.915129 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 02:10:40.920760 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.915257 2561 server.go:962] "Client rotation is on, will bootstrap in background" Sep 09 02:10:40.920760 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.917803 2561 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 09 02:10:40.920760 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.918689 2561 server.go:1019] "Starting client certificate rotation" Sep 09 02:10:40.920760 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.918788 2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 09 02:10:40.920760 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.918826 2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 09 02:10:40.921131 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.921114 2561 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 09 02:10:40.923126 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.923105 2561 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 09 02:10:40.933192 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.933175 2561 log.go:25] "Validated CRI v1 runtime API" Sep 09 02:10:40.935935 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.935922 2561 log.go:25] "Validated CRI v1 image API" Sep 09 02:10:40.936731 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.936715 2561 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 09 02:10:40.944661 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.944641 2561 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 90a7f9c1-cc5c-4617-a5ec-3980d8af7e11:/dev/nvme0n1p4 d1f39813-932e-4f80-b318-2bc929f19d44:/dev/nvme0n1p3] Sep 09 02:10:40.944722 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.944661 2561 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 09 02:10:40.951305 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.951186 2561 manager.go:216] Machine: {Timestamp:2026-09-09 02:10:40.949868356 +0000 UTC m=+0.324899755 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3074657 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23cf6b25551545cc40a57d57896ff0 SystemUUID:ec23cf6b-2555-1545-cc40-a57d57896ff0 BootID:31da9fba-d517-4c27-83cf-8c9f741d1724 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:7e:da:43:61:3f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:7e:da:43:61:3f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:de:f9:bb:cb:97:80 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 09 02:10:40.951305 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.951292 2561 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 09 02:10:40.951452 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.951388 2561 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 09 02:10:40.954041 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.954005 2561 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 09 02:10:40.954175 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.954044 2561 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-218.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 09 02:10:40.955371 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.955354 2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 09 02:10:40.955406 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.955388 2561 topology_manager.go:138] "Creating topology manager with none policy" Sep 09 02:10:40.955406 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.955399 2561 container_manager_linux.go:309] "Creating device plugin manager" Sep 09 02:10:40.955459 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.955407 2561 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 09 02:10:40.956100 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.956088 2561 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 09 02:10:40.957263 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.957252 2561 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 09 02:10:40.961298 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.961286 2561 state_mem.go:36] "Initialized new in-memory state store" Sep 09 02:10:40.961416 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.961407 2561 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 09 02:10:40.961449 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.961439 2561 kubelet.go:486] "Attempting to sync node with API server" Sep 09 02:10:40.961476 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.961451 2561 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 09 02:10:40.961476 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.961462 2561 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 09 02:10:40.961476 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.961474 2561 kubelet.go:398] "Adding apiserver pod source" Sep 09 02:10:40.961561 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.961484 2561 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 09 02:10:40.961877 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.961865 2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 09 02:10:40.961919 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.961888 2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 09 02:10:40.964340 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.964311 2561 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 09 02:10:40.967062 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.967046 2561 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 09 02:10:40.967152 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.967077 2561 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 09 02:10:40.968470 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.968452 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 09 02:10:40.968556 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.968476 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 09 02:10:40.968556 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.968486 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 09 02:10:40.968556 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.968495 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 09 02:10:40.968556 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.968506 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 09 02:10:40.968556 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.968517 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 09 02:10:40.968556 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.968535 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 09 02:10:40.968556 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.968541 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 09 02:10:40.968556 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.968548 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 09 02:10:40.968556 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.968554 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 09 02:10:40.968780 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.968562 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 09 02:10:40.968780 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.968567 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 09 02:10:40.969367 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.969351 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 09 02:10:40.969412 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.969370 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 09 02:10:40.971608 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.971572 2561 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-218.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 09 02:10:40.971670 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:40.971646 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-218.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 09 02:10:40.971711 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:40.971655 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 09 02:10:40.973263 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.973251 2561 server.go:1268] "Started kubelet" Sep 09 02:10:40.973369 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.973306 2561 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 09 02:10:40.973429 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.973351 2561 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 09 02:10:40.973429 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.973421 2561 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 09 02:10:40.974076 ip-10-0-128-218 systemd[1]: Started Kubernetes Kubelet. Sep 09 02:10:40.974638 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.974621 2561 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 09 02:10:40.976644 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.976288 2561 server.go:310] "Adding debug handlers to kubelet server" Sep 09 02:10:40.978065 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:40.978045 2561 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 09 02:10:40.980944 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.980928 2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 09 02:10:40.981472 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.981457 2561 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 09 02:10:40.981626 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:40.981609 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-218.ec2.internal\" not found" Sep 09 02:10:40.982361 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.982343 2561 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 09 02:10:40.982361 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.982360 2561 factory.go:55] Registering systemd factory Sep 09 02:10:40.982361 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.982366 2561 factory.go:223] Registration of the systemd container factory successfully Sep 09 02:10:40.982550 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.982457 2561 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 09 02:10:40.982550 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.982460 2561 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 09 02:10:40.982550 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.982480 2561 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 09 02:10:40.982550 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.982536 2561 reconstruct.go:97] "Volume reconstruction finished" Sep 09 02:10:40.982550 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.982544 2561 reconciler.go:29] "Reconciler: start to sync state" Sep 09 02:10:40.982781 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.982583 2561 factory.go:153] Registering CRI-O factory Sep 09 02:10:40.982781 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.982596 2561 factory.go:223] Registration of the crio container factory successfully Sep 09 02:10:40.982781 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.982613 2561 factory.go:103] Registering Raw factory Sep 09 02:10:40.982781 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.982629 2561 manager.go:1196] Started watching for new ooms in manager Sep 09 02:10:40.983159 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.983146 2561 manager.go:349] Starting recovery of all containers Sep 09 02:10:40.985953 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.985928 2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wn8s6" Sep 09 02:10:40.987148 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:40.987119 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 09 02:10:40.987277 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:40.987254 2561 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-218.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 09 02:10:40.991382 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:40.986982 2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-218.ec2.internal.18d3851e04db5d34 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-218.ec2.internal,UID:ip-10-0-128-218.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-218.ec2.internal,},FirstTimestamp:2026-09-09 02:10:40.973217076 +0000 UTC m=+0.348248477,LastTimestamp:2026-09-09 02:10:40.973217076 +0000 UTC m=+0.348248477,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-218.ec2.internal,}" Sep 09 02:10:40.993000 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.992978 2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wn8s6" Sep 09 02:10:40.995186 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:40.995168 2561 manager.go:354] Recovery completed Sep 09 02:10:41.000787 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.000772 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 02:10:41.002850 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.002830 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-218.ec2.internal" event="NodeHasSufficientMemory" Sep 09 02:10:41.002929 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.002886 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-218.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 02:10:41.002929 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.002901 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-218.ec2.internal" event="NodeHasSufficientPID" Sep 09 02:10:41.003556 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.003540 2561 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 09 02:10:41.003556 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.003549 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 02:10:41.003556 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.003555 2561 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 09 02:10:41.003713 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.003571 2561 state_mem.go:36] "Initialized new in-memory state store" Sep 09 02:10:41.004236 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.004223 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-218.ec2.internal" event="NodeHasSufficientMemory" Sep 09 02:10:41.004295 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.004250 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-218.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 02:10:41.004295 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.004263 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-218.ec2.internal" event="NodeHasSufficientPID" Sep 09 02:10:41.005376 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:41.005305 2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-218.ec2.internal.18d3851e069f8042 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-218.ec2.internal,UID:ip-10-0-128-218.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-218.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-218.ec2.internal,},FirstTimestamp:2026-09-09 02:10:41.002848322 +0000 UTC m=+0.377879722,LastTimestamp:2026-09-09 02:10:41.002848322 +0000 UTC m=+0.377879722,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-218.ec2.internal,}" Sep 09 02:10:41.005611 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.005599 2561 policy_none.go:49] "None policy: Start" Sep 09 02:10:41.005656 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.005616 2561 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 09 02:10:41.005656 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.005626 2561 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 09 02:10:41.007583 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.007572 2561 policy_none.go:47] "Start" Sep 09 02:10:41.034671 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.034653 2561 manager.go:337] "Starting Device Plugin manager" Sep 09 02:10:41.060687 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:41.034697 2561 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 09 02:10:41.060687 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.034711 2561 server.go:87] "Starting device plugin registration server" Sep 09 02:10:41.060687 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.035072 2561 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 09 02:10:41.060687 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.035083 2561 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 09 02:10:41.060687 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.035211 2561 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 09 02:10:41.060687 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.035288 2561 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 09 02:10:41.060687 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.035294 2561 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 09 02:10:41.060687 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:41.035876 2561 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 09 02:10:41.060687 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:41.035918 2561 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-218.ec2.internal\" not found" Sep 09 02:10:41.103056 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.102993 2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 09 02:10:41.104291 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.104273 2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 09 02:10:41.104370 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.104329 2561 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 09 02:10:41.104370 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.104353 2561 kubelet.go:2448] "Starting kubelet main sync loop" Sep 09 02:10:41.104484 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:41.104461 2561 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 09 02:10:41.106933 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.106907 2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 02:10:41.135422 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.135398 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 02:10:41.136444 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.136425 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-218.ec2.internal" event="NodeHasSufficientMemory" Sep 09 02:10:41.136556 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.136454 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-218.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 02:10:41.136556 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.136467 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-218.ec2.internal" event="NodeHasSufficientPID" Sep 09 02:10:41.136556 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.136495 2561 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-218.ec2.internal" Sep 09 02:10:41.142756 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.142740 2561 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-218.ec2.internal" Sep 09 02:10:41.142828 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:41.142765 2561 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-218.ec2.internal\": node \"ip-10-0-128-218.ec2.internal\" not found" Sep 09 02:10:41.159763 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:41.159677 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-218.ec2.internal\" not found" Sep 09 02:10:41.204907 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.204873 2561 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-218.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-218.ec2.internal"] Sep 09 02:10:41.205074 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.204947 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 02:10:41.206526 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.206508 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-218.ec2.internal" event="NodeHasSufficientMemory" Sep 09 02:10:41.206623 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.206536 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-218.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 02:10:41.206623 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.206546 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-218.ec2.internal" event="NodeHasSufficientPID" Sep 09 02:10:41.208858 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.208844 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 02:10:41.208974 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.208956 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-218.ec2.internal" Sep 09 02:10:41.209044 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.208984 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 02:10:41.209596 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.209582 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-218.ec2.internal" event="NodeHasSufficientMemory" Sep 09 02:10:41.209657 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.209607 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-218.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 02:10:41.209657 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.209606 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-218.ec2.internal" event="NodeHasSufficientMemory" Sep 09 02:10:41.209657 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.209617 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-218.ec2.internal" event="NodeHasSufficientPID" Sep 09 02:10:41.209657 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.209626 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-218.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 02:10:41.209657 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.209636 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-218.ec2.internal" event="NodeHasSufficientPID" Sep 09 02:10:41.211803 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.211788 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-218.ec2.internal" Sep 09 02:10:41.211890 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.211813 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 02:10:41.212499 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.212486 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-218.ec2.internal" event="NodeHasSufficientMemory" Sep 09 02:10:41.212573 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.212505 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-218.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 02:10:41.212573 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.212517 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-218.ec2.internal" event="NodeHasSufficientPID" Sep 09 02:10:41.230685 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:41.230658 2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-218.ec2.internal\" not found" node="ip-10-0-128-218.ec2.internal" Sep 09 02:10:41.234760 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:41.234742 2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-218.ec2.internal\" not found" node="ip-10-0-128-218.ec2.internal" Sep 09 02:10:41.260751 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:41.260724 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-218.ec2.internal\" not found" Sep 09 02:10:41.283792 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.283767 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/45b4e2b9f311e3b6046acb668fc1406e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-218.ec2.internal\" (UID: \"45b4e2b9f311e3b6046acb668fc1406e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-218.ec2.internal" Sep 09 02:10:41.283867 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.283799 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/45b4e2b9f311e3b6046acb668fc1406e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-218.ec2.internal\" (UID: \"45b4e2b9f311e3b6046acb668fc1406e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-218.ec2.internal" Sep 09 02:10:41.283867 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.283816 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fe8949b5a0140471a9c7b59c781130ce-config\") pod \"kube-apiserver-proxy-ip-10-0-128-218.ec2.internal\" (UID: \"fe8949b5a0140471a9c7b59c781130ce\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-218.ec2.internal" Sep 09 02:10:41.361657 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:41.361624 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-218.ec2.internal\" not found" Sep 09 02:10:41.383959 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.383933 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/45b4e2b9f311e3b6046acb668fc1406e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-218.ec2.internal\" (UID: \"45b4e2b9f311e3b6046acb668fc1406e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-218.ec2.internal" Sep 09 02:10:41.384102 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.383966 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fe8949b5a0140471a9c7b59c781130ce-config\") pod \"kube-apiserver-proxy-ip-10-0-128-218.ec2.internal\" (UID: \"fe8949b5a0140471a9c7b59c781130ce\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-218.ec2.internal" Sep 09 02:10:41.384102 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.383989 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/45b4e2b9f311e3b6046acb668fc1406e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-218.ec2.internal\" (UID: \"45b4e2b9f311e3b6046acb668fc1406e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-218.ec2.internal" Sep 09 02:10:41.384102 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.384055 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/45b4e2b9f311e3b6046acb668fc1406e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-218.ec2.internal\" (UID: \"45b4e2b9f311e3b6046acb668fc1406e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-218.ec2.internal" Sep 09 02:10:41.384102 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.384090 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fe8949b5a0140471a9c7b59c781130ce-config\") pod \"kube-apiserver-proxy-ip-10-0-128-218.ec2.internal\" (UID: \"fe8949b5a0140471a9c7b59c781130ce\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-218.ec2.internal" Sep 09 02:10:41.384102 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.384091 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/45b4e2b9f311e3b6046acb668fc1406e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-218.ec2.internal\" (UID: \"45b4e2b9f311e3b6046acb668fc1406e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-218.ec2.internal" Sep 09 02:10:41.462320 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:41.462235 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-218.ec2.internal\" not found" Sep 09 02:10:41.533749 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.533702 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-218.ec2.internal" Sep 09 02:10:41.538740 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.538633 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-218.ec2.internal" Sep 09 02:10:41.563194 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:41.563174 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-218.ec2.internal\" not found" Sep 09 02:10:41.669392 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:41.669343 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-218.ec2.internal\" not found" Sep 09 02:10:41.775966 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:41.775885 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-218.ec2.internal\" not found" Sep 09 02:10:41.877117 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:41.877070 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-218.ec2.internal\" not found" Sep 09 02:10:41.919697 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.919660 2561 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 09 02:10:41.920194 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.919820 2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 09 02:10:41.977797 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:41.977759 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-218.ec2.internal\" not found" Sep 09 02:10:41.999203 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.999163 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-08 02:05:41 +0000 UTC" deadline="2028-03-04 23:20:24.692011997 +0000 UTC" Sep 09 02:10:41.999203 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:41.999195 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13029h9m42.692821598s" Sep 09 02:10:42.053171 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.053082 2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 02:10:42.078138 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:42.078105 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-218.ec2.internal\" not found" Sep 09 02:10:42.171604 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:42.171573 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe8949b5a0140471a9c7b59c781130ce.slice/crio-54dec18b43f7daae219a5886e6747bd06e14bbf54ef1e0bbc4656b3c3daddbfa WatchSource:0}: Error finding container 54dec18b43f7daae219a5886e6747bd06e14bbf54ef1e0bbc4656b3c3daddbfa: Status 404 returned error can't find the container with id 54dec18b43f7daae219a5886e6747bd06e14bbf54ef1e0bbc4656b3c3daddbfa Sep 09 02:10:42.171885 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:42.171862 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45b4e2b9f311e3b6046acb668fc1406e.slice/crio-8fcdb17a45103964102f86ff4c6acaef8a1641ef750c6cdf53374e3f48318077 WatchSource:0}: Error finding container 8fcdb17a45103964102f86ff4c6acaef8a1641ef750c6cdf53374e3f48318077: Status 404 returned error can't find the container with id 8fcdb17a45103964102f86ff4c6acaef8a1641ef750c6cdf53374e3f48318077 Sep 09 02:10:42.178260 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:42.178225 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-218.ec2.internal\" not found" Sep 09 02:10:42.178780 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.178765 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 02:10:42.278758 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:42.278711 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-218.ec2.internal\" not found" Sep 09 02:10:42.379804 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:42.379732 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-218.ec2.internal\" not found" Sep 09 02:10:42.480548 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:42.480519 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-218.ec2.internal\" not found" Sep 09 02:10:42.510139 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.510108 2561 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 02:10:42.516440 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.516410 2561 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 02:10:42.585447 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.585412 2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-218.ec2.internal" Sep 09 02:10:42.597073 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.597032 2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 09 02:10:42.598116 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.598100 2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-218.ec2.internal" Sep 09 02:10:42.605067 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.605050 2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 09 02:10:42.963225 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.963193 2561 apiserver.go:52] "Watching apiserver" Sep 09 02:10:42.971205 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.971177 2561 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 09 02:10:42.972668 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.972403 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-z7f6v","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-218.ec2.internal","openshift-multus/multus-additional-cni-plugins-5g4kk","openshift-network-diagnostics/network-check-target-4mzjj","openshift-network-operator/iptables-alerter-dt57p","kube-system/kube-apiserver-proxy-ip-10-0-128-218.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tvbf","openshift-cluster-node-tuning-operator/tuned-rffj8","openshift-multus/multus-psv7l","openshift-multus/network-metrics-daemon-8x9lh","openshift-ovn-kubernetes/ovnkube-node-9cxcz","kube-system/global-pull-secret-syncer-mzt99","kube-system/konnectivity-agent-g8jhc","openshift-dns/node-resolver-wvtrf"] Sep 09 02:10:42.975731 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.975215 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tvbf" Sep 09 02:10:42.978690 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.977723 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g8jhc" Sep 09 02:10:42.978690 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.977893 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-c6knf\"" Sep 09 02:10:42.978690 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.978140 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 09 02:10:42.978690 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.978367 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 09 02:10:42.978690 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.978540 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 09 02:10:42.982470 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.980233 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 09 02:10:42.982470 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.980574 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 09 02:10:42.982470 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.980578 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9r2jl\"" Sep 09 02:10:42.982470 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.980757 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wvtrf" Sep 09 02:10:42.982470 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.981953 2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 09 02:10:42.982910 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.982887 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4mzjj" Sep 09 02:10:42.983019 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:42.982952 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4mzjj" podUID="ae5099dc-0b2e-4a01-887c-3076a88ecbd8" Sep 09 02:10:42.983711 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.983677 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 09 02:10:42.983804 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.983712 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 09 02:10:42.984194 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.984040 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-lstn7\"" Sep 09 02:10:42.984891 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.984871 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dt57p" Sep 09 02:10:42.986870 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.986844 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 09 02:10:42.988965 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.987198 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 09 02:10:42.988965 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.987536 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jmk4b\"" Sep 09 02:10:42.988965 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.987729 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 09 02:10:42.988965 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.988236 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:42.989906 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.989885 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-psv7l" Sep 09 02:10:42.993807 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.991610 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ntgj4\"" Sep 09 02:10:42.993807 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.991831 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 09 02:10:42.993807 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.992015 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 09 02:10:42.993807 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.992323 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/79f5d659-762b-4f17-be10-469a57ff032b-agent-certs\") pod \"konnectivity-agent-g8jhc\" (UID: \"79f5d659-762b-4f17-be10-469a57ff032b\") " pod="kube-system/konnectivity-agent-g8jhc" Sep 09 02:10:42.993807 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.992353 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2qvp\" (UniqueName: \"kubernetes.io/projected/55dc8777-be96-4d4f-9f9f-8180a1e158fe-kube-api-access-w2qvp\") pod \"node-resolver-wvtrf\" (UID: \"55dc8777-be96-4d4f-9f9f-8180a1e158fe\") " pod="openshift-dns/node-resolver-wvtrf" Sep 09 02:10:42.993807 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.992384 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4974aeea-5424-46f6-b987-2eea8d42f810-iptables-alerter-script\") pod \"iptables-alerter-dt57p\" (UID: \"4974aeea-5424-46f6-b987-2eea8d42f810\") " pod="openshift-network-operator/iptables-alerter-dt57p" Sep 09 02:10:42.993807 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.992411 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8tvbf\" (UID: \"09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tvbf" Sep 09 02:10:42.993807 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.992437 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea-device-dir\") pod \"aws-ebs-csi-driver-node-8tvbf\" (UID: \"09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tvbf" Sep 09 02:10:42.993807 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.992460 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4456s\" (UniqueName: \"kubernetes.io/projected/ae5099dc-0b2e-4a01-887c-3076a88ecbd8-kube-api-access-4456s\") pod \"network-check-target-4mzjj\" (UID: \"ae5099dc-0b2e-4a01-887c-3076a88ecbd8\") " pod="openshift-network-diagnostics/network-check-target-4mzjj" Sep 09 02:10:42.993807 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.992486 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m98xp\" (UniqueName: \"kubernetes.io/projected/4974aeea-5424-46f6-b987-2eea8d42f810-kube-api-access-m98xp\") pod \"iptables-alerter-dt57p\" (UID: \"4974aeea-5424-46f6-b987-2eea8d42f810\") " pod="openshift-network-operator/iptables-alerter-dt57p" Sep 09 02:10:42.993807 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.992507 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea-etc-selinux\") pod \"aws-ebs-csi-driver-node-8tvbf\" (UID: \"09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tvbf" Sep 09 02:10:42.993807 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.992524 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whrp8\" (UniqueName: \"kubernetes.io/projected/09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea-kube-api-access-whrp8\") pod \"aws-ebs-csi-driver-node-8tvbf\" (UID: \"09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tvbf" Sep 09 02:10:42.993807 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.992556 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55dc8777-be96-4d4f-9f9f-8180a1e158fe-tmp-dir\") pod \"node-resolver-wvtrf\" (UID: \"55dc8777-be96-4d4f-9f9f-8180a1e158fe\") " pod="openshift-dns/node-resolver-wvtrf" Sep 09 02:10:42.993807 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.992579 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4974aeea-5424-46f6-b987-2eea8d42f810-host-slash\") pod \"iptables-alerter-dt57p\" (UID: \"4974aeea-5424-46f6-b987-2eea8d42f810\") " pod="openshift-network-operator/iptables-alerter-dt57p" Sep 09 02:10:42.993807 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.992616 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea-socket-dir\") pod \"aws-ebs-csi-driver-node-8tvbf\" (UID: \"09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tvbf" Sep 09 02:10:42.993807 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.992634 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea-registration-dir\") pod \"aws-ebs-csi-driver-node-8tvbf\" (UID: \"09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tvbf" Sep 09 02:10:42.993807 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.992657 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/79f5d659-762b-4f17-be10-469a57ff032b-konnectivity-ca\") pod \"konnectivity-agent-g8jhc\" (UID: \"79f5d659-762b-4f17-be10-469a57ff032b\") " pod="kube-system/konnectivity-agent-g8jhc" Sep 09 02:10:42.994677 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.992673 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/55dc8777-be96-4d4f-9f9f-8180a1e158fe-hosts-file\") pod \"node-resolver-wvtrf\" (UID: \"55dc8777-be96-4d4f-9f9f-8180a1e158fe\") " pod="openshift-dns/node-resolver-wvtrf" Sep 09 02:10:42.994677 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.992688 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea-sys-fs\") pod \"aws-ebs-csi-driver-node-8tvbf\" (UID: \"09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tvbf" Sep 09 02:10:42.994677 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.993079 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-z7f6v" Sep 09 02:10:42.994677 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.993120 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 09 02:10:42.997712 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.995958 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 09 02:10:42.997712 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.996027 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 09 02:10:42.997712 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.996192 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8x9lh" Sep 09 02:10:42.997712 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:42.996287 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8x9lh" podUID="a06a9649-a313-401a-80cb-c71fddc9d349" Sep 09 02:10:42.997712 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.996331 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-gzr6v\"" Sep 09 02:10:42.997712 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.996393 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 09 02:10:42.998265 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.998243 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 09 02:10:42.998684 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.998391 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 09 02:10:42.998684 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.998437 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 09 02:10:42.998684 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.998504 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lvfds\"" Sep 09 02:10:42.999646 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:42.999626 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.003273 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.003240 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mzt99" Sep 09 02:10:43.003356 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:43.003307 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mzt99" podUID="0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129" Sep 09 02:10:43.004909 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.004889 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5g4kk" Sep 09 02:10:43.007544 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.007521 2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 09 02:10:43.007636 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.007571 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 09 02:10:43.007890 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.007873 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 09 02:10:43.009861 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.009633 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 09 02:10:43.009861 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.009652 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 09 02:10:43.009861 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.009719 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 09 02:10:43.010523 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.010479 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8dc6t\"" Sep 09 02:10:43.010767 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.010750 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zxj2l\"" Sep 09 02:10:43.011139 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.011115 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 09 02:10:43.012324 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.012303 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 09 02:10:43.012564 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.012547 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 09 02:10:43.066619 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.066585 2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-nw8jp" Sep 09 02:10:43.082875 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.082847 2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-nw8jp" Sep 09 02:10:43.083640 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.083621 2561 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 09 02:10:43.092848 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.092825 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8tvbf\" (UID: \"09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tvbf" Sep 09 02:10:43.092976 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.092864 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-host-var-lib-cni-bin\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.092976 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.092920 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/84536d51-1f06-4aac-ba98-7773a0e01692-serviceca\") pod \"node-ca-z7f6v\" (UID: \"84536d51-1f06-4aac-ba98-7773a0e01692\") " pod="openshift-image-registry/node-ca-z7f6v" Sep 09 02:10:43.092976 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.092947 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpsfv\" (UniqueName: \"kubernetes.io/projected/b3e07c44-d299-4e8d-b240-0988e28e9c2b-kube-api-access-wpsfv\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.092976 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.092971 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-run-systemd\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.093220 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.092994 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-log-socket\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.093220 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.093020 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-multus-cni-dir\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.093220 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.092993 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8tvbf\" (UID: \"09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tvbf" Sep 09 02:10:43.093220 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.093048 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea-registration-dir\") pod \"aws-ebs-csi-driver-node-8tvbf\" (UID: \"09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tvbf" Sep 09 02:10:43.093220 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.093104 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a06a9649-a313-401a-80cb-c71fddc9d349-metrics-certs\") pod \"network-metrics-daemon-8x9lh\" (UID: \"a06a9649-a313-401a-80cb-c71fddc9d349\") " pod="openshift-multus/network-metrics-daemon-8x9lh" Sep 09 02:10:43.093220 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.093120 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea-registration-dir\") pod \"aws-ebs-csi-driver-node-8tvbf\" (UID: \"09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tvbf" Sep 09 02:10:43.093220 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.093132 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-host-cni-bin\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.093220 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.093173 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-host-cni-netd\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.093220 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.093201 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/85d4dd2e-a53d-455e-a542-9ce5ebd5b479-system-cni-dir\") pod \"multus-additional-cni-plugins-5g4kk\" (UID: \"85d4dd2e-a53d-455e-a542-9ce5ebd5b479\") " pod="openshift-multus/multus-additional-cni-plugins-5g4kk" Sep 09 02:10:43.093616 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.093227 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b3e07c44-d299-4e8d-b240-0988e28e9c2b-etc-modprobe-d\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.093616 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.093251 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b3e07c44-d299-4e8d-b240-0988e28e9c2b-etc-sysctl-d\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.093616 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.093300 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdhk5\" (UniqueName: \"kubernetes.io/projected/a06a9649-a313-401a-80cb-c71fddc9d349-kube-api-access-jdhk5\") pod \"network-metrics-daemon-8x9lh\" (UID: \"a06a9649-a313-401a-80cb-c71fddc9d349\") " pod="openshift-multus/network-metrics-daemon-8x9lh" Sep 09 02:10:43.093616 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.093326 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-run-openvswitch\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.093616 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.093361 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/85d4dd2e-a53d-455e-a542-9ce5ebd5b479-cni-binary-copy\") pod \"multus-additional-cni-plugins-5g4kk\" (UID: \"85d4dd2e-a53d-455e-a542-9ce5ebd5b479\") " pod="openshift-multus/multus-additional-cni-plugins-5g4kk" Sep 09 02:10:43.093616 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.093403 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b3e07c44-d299-4e8d-b240-0988e28e9c2b-etc-systemd\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.093616 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.093437 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4456s\" (UniqueName: \"kubernetes.io/projected/ae5099dc-0b2e-4a01-887c-3076a88ecbd8-kube-api-access-4456s\") pod \"network-check-target-4mzjj\" (UID: \"ae5099dc-0b2e-4a01-887c-3076a88ecbd8\") " pod="openshift-network-diagnostics/network-check-target-4mzjj" Sep 09 02:10:43.093616 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.093463 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-etc-openvswitch\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.093616 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.093483 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1b49c337-0256-42b7-9c89-1ceea7cbe801-ovn-node-metrics-cert\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.093616 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.093528 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-multus-socket-dir-parent\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.093616 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.093553 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqgkn\" (UniqueName: \"kubernetes.io/projected/7aaa0ef6-52d4-4878-9a87-8cc0da967919-kube-api-access-kqgkn\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.093616 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.093584 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129-dbus\") pod \"global-pull-secret-syncer-mzt99\" (UID: \"0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129\") " pod="kube-system/global-pull-secret-syncer-mzt99" Sep 09 02:10:43.094085 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.093628 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m98xp\" (UniqueName: \"kubernetes.io/projected/4974aeea-5424-46f6-b987-2eea8d42f810-kube-api-access-m98xp\") pod \"iptables-alerter-dt57p\" (UID: \"4974aeea-5424-46f6-b987-2eea8d42f810\") " pod="openshift-network-operator/iptables-alerter-dt57p" Sep 09 02:10:43.094085 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.093653 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-host-run-netns\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.094085 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.093680 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/85d4dd2e-a53d-455e-a542-9ce5ebd5b479-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5g4kk\" (UID: \"85d4dd2e-a53d-455e-a542-9ce5ebd5b479\") " pod="openshift-multus/multus-additional-cni-plugins-5g4kk" Sep 09 02:10:43.094085 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.093723 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-host-var-lib-cni-multus\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.094085 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.093748 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b3e07c44-d299-4e8d-b240-0988e28e9c2b-etc-tuned\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.094085 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.093773 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4974aeea-5424-46f6-b987-2eea8d42f810-host-slash\") pod \"iptables-alerter-dt57p\" (UID: \"4974aeea-5424-46f6-b987-2eea8d42f810\") " pod="openshift-network-operator/iptables-alerter-dt57p" Sep 09 02:10:43.094085 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.093798 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/79f5d659-762b-4f17-be10-469a57ff032b-konnectivity-ca\") pod \"konnectivity-agent-g8jhc\" (UID: \"79f5d659-762b-4f17-be10-469a57ff032b\") " pod="kube-system/konnectivity-agent-g8jhc" Sep 09 02:10:43.094085 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.093825 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-host-run-netns\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.094085 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.093850 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1b49c337-0256-42b7-9c89-1ceea7cbe801-ovnkube-script-lib\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.094085 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.093874 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-host-run-k8s-cni-cncf-io\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.094085 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.093894 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4974aeea-5424-46f6-b987-2eea8d42f810-host-slash\") pod \"iptables-alerter-dt57p\" (UID: \"4974aeea-5424-46f6-b987-2eea8d42f810\") " pod="openshift-network-operator/iptables-alerter-dt57p" Sep 09 02:10:43.094085 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.093897 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-multus-conf-dir\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.094085 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.093935 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b3e07c44-d299-4e8d-b240-0988e28e9c2b-etc-sysconfig\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.094085 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.093965 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b3e07c44-d299-4e8d-b240-0988e28e9c2b-run\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.094085 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.094002 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-host-run-ovn-kubernetes\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.094085 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.094039 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1b49c337-0256-42b7-9c89-1ceea7cbe801-ovnkube-config\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.094085 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.094084 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1b49c337-0256-42b7-9c89-1ceea7cbe801-env-overrides\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.094849 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.094123 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/85d4dd2e-a53d-455e-a542-9ce5ebd5b479-cnibin\") pod \"multus-additional-cni-plugins-5g4kk\" (UID: \"85d4dd2e-a53d-455e-a542-9ce5ebd5b479\") " pod="openshift-multus/multus-additional-cni-plugins-5g4kk" Sep 09 02:10:43.094849 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.094154 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-hostroot\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.094849 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.094180 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndzs7\" (UniqueName: \"kubernetes.io/projected/84536d51-1f06-4aac-ba98-7773a0e01692-kube-api-access-ndzs7\") pod \"node-ca-z7f6v\" (UID: \"84536d51-1f06-4aac-ba98-7773a0e01692\") " pod="openshift-image-registry/node-ca-z7f6v" Sep 09 02:10:43.094849 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.094201 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea-etc-selinux\") pod \"aws-ebs-csi-driver-node-8tvbf\" (UID: \"09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tvbf" Sep 09 02:10:43.094849 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.094224 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-host-kubelet\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.094849 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.094244 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-host-slash\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.094849 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.094278 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea-etc-selinux\") pod \"aws-ebs-csi-driver-node-8tvbf\" (UID: \"09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tvbf" Sep 09 02:10:43.094849 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.094285 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bg7s8\" (UniqueName: \"kubernetes.io/projected/1b49c337-0256-42b7-9c89-1ceea7cbe801-kube-api-access-bg7s8\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.094849 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.094327 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7aaa0ef6-52d4-4878-9a87-8cc0da967919-multus-daemon-config\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.094849 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.094356 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-etc-kubernetes\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.094849 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.094382 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b3e07c44-d299-4e8d-b240-0988e28e9c2b-host\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.094849 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.094412 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/55dc8777-be96-4d4f-9f9f-8180a1e158fe-hosts-file\") pod \"node-resolver-wvtrf\" (UID: \"55dc8777-be96-4d4f-9f9f-8180a1e158fe\") " pod="openshift-dns/node-resolver-wvtrf" Sep 09 02:10:43.094849 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.094449 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-node-log\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.094849 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.094472 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/55dc8777-be96-4d4f-9f9f-8180a1e158fe-hosts-file\") pod \"node-resolver-wvtrf\" (UID: \"55dc8777-be96-4d4f-9f9f-8180a1e158fe\") " pod="openshift-dns/node-resolver-wvtrf" Sep 09 02:10:43.094849 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.094476 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-host-var-lib-kubelet\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.094849 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.094508 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/84536d51-1f06-4aac-ba98-7773a0e01692-host\") pod \"node-ca-z7f6v\" (UID: \"84536d51-1f06-4aac-ba98-7773a0e01692\") " pod="openshift-image-registry/node-ca-z7f6v" Sep 09 02:10:43.094849 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.094538 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3e07c44-d299-4e8d-b240-0988e28e9c2b-var-lib-kubelet\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.095597 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.094563 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b3e07c44-d299-4e8d-b240-0988e28e9c2b-tmp\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.095597 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.094591 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea-sys-fs\") pod \"aws-ebs-csi-driver-node-8tvbf\" (UID: \"09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tvbf" Sep 09 02:10:43.095597 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.094634 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea-sys-fs\") pod \"aws-ebs-csi-driver-node-8tvbf\" (UID: \"09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tvbf" Sep 09 02:10:43.095597 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.094640 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/79f5d659-762b-4f17-be10-469a57ff032b-agent-certs\") pod \"konnectivity-agent-g8jhc\" (UID: \"79f5d659-762b-4f17-be10-469a57ff032b\") " pod="kube-system/konnectivity-agent-g8jhc" Sep 09 02:10:43.095597 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.094694 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2qvp\" (UniqueName: \"kubernetes.io/projected/55dc8777-be96-4d4f-9f9f-8180a1e158fe-kube-api-access-w2qvp\") pod \"node-resolver-wvtrf\" (UID: \"55dc8777-be96-4d4f-9f9f-8180a1e158fe\") " pod="openshift-dns/node-resolver-wvtrf" Sep 09 02:10:43.095597 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.094721 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-run-ovn\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.095597 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.094740 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/79f5d659-762b-4f17-be10-469a57ff032b-konnectivity-ca\") pod \"konnectivity-agent-g8jhc\" (UID: \"79f5d659-762b-4f17-be10-469a57ff032b\") " pod="kube-system/konnectivity-agent-g8jhc" Sep 09 02:10:43.095597 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.094763 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/85d4dd2e-a53d-455e-a542-9ce5ebd5b479-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5g4kk\" (UID: \"85d4dd2e-a53d-455e-a542-9ce5ebd5b479\") " pod="openshift-multus/multus-additional-cni-plugins-5g4kk" Sep 09 02:10:43.095597 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.094817 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-host-run-multus-certs\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.095597 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.094867 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea-device-dir\") pod \"aws-ebs-csi-driver-node-8tvbf\" (UID: \"09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tvbf" Sep 09 02:10:43.095597 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.094894 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3e07c44-d299-4e8d-b240-0988e28e9c2b-etc-kubernetes\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.095597 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.094913 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whrp8\" (UniqueName: \"kubernetes.io/projected/09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea-kube-api-access-whrp8\") pod \"aws-ebs-csi-driver-node-8tvbf\" (UID: \"09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tvbf" Sep 09 02:10:43.095597 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.094929 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55dc8777-be96-4d4f-9f9f-8180a1e158fe-tmp-dir\") pod \"node-resolver-wvtrf\" (UID: \"55dc8777-be96-4d4f-9f9f-8180a1e158fe\") " pod="openshift-dns/node-resolver-wvtrf" Sep 09 02:10:43.095597 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.094952 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.095597 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.094988 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/85d4dd2e-a53d-455e-a542-9ce5ebd5b479-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5g4kk\" (UID: \"85d4dd2e-a53d-455e-a542-9ce5ebd5b479\") " pod="openshift-multus/multus-additional-cni-plugins-5g4kk" Sep 09 02:10:43.095597 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.095015 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-cnibin\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.095597 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.095031 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea-device-dir\") pod \"aws-ebs-csi-driver-node-8tvbf\" (UID: \"09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tvbf" Sep 09 02:10:43.096350 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.095060 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7aaa0ef6-52d4-4878-9a87-8cc0da967919-cni-binary-copy\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.096350 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.095085 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129-kubelet-config\") pod \"global-pull-secret-syncer-mzt99\" (UID: \"0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129\") " pod="kube-system/global-pull-secret-syncer-mzt99" Sep 09 02:10:43.096350 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.095111 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea-socket-dir\") pod \"aws-ebs-csi-driver-node-8tvbf\" (UID: \"09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tvbf" Sep 09 02:10:43.096350 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.095134 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-systemd-units\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.096350 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.095169 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-var-lib-openvswitch\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.096350 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.095197 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129-original-pull-secret\") pod \"global-pull-secret-syncer-mzt99\" (UID: \"0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129\") " pod="kube-system/global-pull-secret-syncer-mzt99" Sep 09 02:10:43.096350 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.095228 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b3e07c44-d299-4e8d-b240-0988e28e9c2b-etc-sysctl-conf\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.096350 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.095248 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea-socket-dir\") pod \"aws-ebs-csi-driver-node-8tvbf\" (UID: \"09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tvbf" Sep 09 02:10:43.096350 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.095252 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b3e07c44-d299-4e8d-b240-0988e28e9c2b-sys\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.096350 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.095284 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b3e07c44-d299-4e8d-b240-0988e28e9c2b-lib-modules\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.096350 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.095298 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55dc8777-be96-4d4f-9f9f-8180a1e158fe-tmp-dir\") pod \"node-resolver-wvtrf\" (UID: \"55dc8777-be96-4d4f-9f9f-8180a1e158fe\") " pod="openshift-dns/node-resolver-wvtrf" Sep 09 02:10:43.096350 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.095308 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8dzv\" (UniqueName: \"kubernetes.io/projected/85d4dd2e-a53d-455e-a542-9ce5ebd5b479-kube-api-access-z8dzv\") pod \"multus-additional-cni-plugins-5g4kk\" (UID: \"85d4dd2e-a53d-455e-a542-9ce5ebd5b479\") " pod="openshift-multus/multus-additional-cni-plugins-5g4kk" Sep 09 02:10:43.096350 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.095340 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-system-cni-dir\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.096350 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.095382 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4974aeea-5424-46f6-b987-2eea8d42f810-iptables-alerter-script\") pod \"iptables-alerter-dt57p\" (UID: \"4974aeea-5424-46f6-b987-2eea8d42f810\") " pod="openshift-network-operator/iptables-alerter-dt57p" Sep 09 02:10:43.096350 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.095492 2561 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 09 02:10:43.096350 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.095854 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4974aeea-5424-46f6-b987-2eea8d42f810-iptables-alerter-script\") pod \"iptables-alerter-dt57p\" (UID: \"4974aeea-5424-46f6-b987-2eea8d42f810\") " pod="openshift-network-operator/iptables-alerter-dt57p" Sep 09 02:10:43.100157 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.100136 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/79f5d659-762b-4f17-be10-469a57ff032b-agent-certs\") pod \"konnectivity-agent-g8jhc\" (UID: \"79f5d659-762b-4f17-be10-469a57ff032b\") " pod="kube-system/konnectivity-agent-g8jhc" Sep 09 02:10:43.104204 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:43.104184 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 02:10:43.104204 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:43.104206 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 02:10:43.104350 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:43.104215 2561 projected.go:196] Error preparing data for projected volume kube-api-access-4456s for pod openshift-network-diagnostics/network-check-target-4mzjj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 02:10:43.104350 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:43.104283 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae5099dc-0b2e-4a01-887c-3076a88ecbd8-kube-api-access-4456s podName:ae5099dc-0b2e-4a01-887c-3076a88ecbd8 nodeName:}" failed. No retries permitted until 2026-09-09 02:10:43.604265208 +0000 UTC m=+2.979296601 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4456s" (UniqueName: "kubernetes.io/projected/ae5099dc-0b2e-4a01-887c-3076a88ecbd8-kube-api-access-4456s") pod "network-check-target-4mzjj" (UID: "ae5099dc-0b2e-4a01-887c-3076a88ecbd8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 02:10:43.108651 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.108632 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m98xp\" (UniqueName: \"kubernetes.io/projected/4974aeea-5424-46f6-b987-2eea8d42f810-kube-api-access-m98xp\") pod \"iptables-alerter-dt57p\" (UID: \"4974aeea-5424-46f6-b987-2eea8d42f810\") " pod="openshift-network-operator/iptables-alerter-dt57p" Sep 09 02:10:43.108787 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.108765 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whrp8\" (UniqueName: \"kubernetes.io/projected/09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea-kube-api-access-whrp8\") pod \"aws-ebs-csi-driver-node-8tvbf\" (UID: \"09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tvbf" Sep 09 02:10:43.111102 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.110816 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-218.ec2.internal" event={"ID":"fe8949b5a0140471a9c7b59c781130ce","Type":"ContainerStarted","Data":"54dec18b43f7daae219a5886e6747bd06e14bbf54ef1e0bbc4656b3c3daddbfa"} Sep 09 02:10:43.112933 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.112892 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-218.ec2.internal" event={"ID":"45b4e2b9f311e3b6046acb668fc1406e","Type":"ContainerStarted","Data":"8fcdb17a45103964102f86ff4c6acaef8a1641ef750c6cdf53374e3f48318077"} Sep 09 02:10:43.113338 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.113319 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2qvp\" (UniqueName: \"kubernetes.io/projected/55dc8777-be96-4d4f-9f9f-8180a1e158fe-kube-api-access-w2qvp\") pod \"node-resolver-wvtrf\" (UID: \"55dc8777-be96-4d4f-9f9f-8180a1e158fe\") " pod="openshift-dns/node-resolver-wvtrf" Sep 09 02:10:43.196564 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.196527 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1b49c337-0256-42b7-9c89-1ceea7cbe801-ovnkube-script-lib\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.196564 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.196569 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-host-run-k8s-cni-cncf-io\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.196829 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.196593 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-multus-conf-dir\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.196829 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.196615 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b3e07c44-d299-4e8d-b240-0988e28e9c2b-etc-sysconfig\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.196829 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.196638 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b3e07c44-d299-4e8d-b240-0988e28e9c2b-run\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.196829 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.196662 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-host-run-ovn-kubernetes\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.196829 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.196669 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-multus-conf-dir\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.196829 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.196685 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1b49c337-0256-42b7-9c89-1ceea7cbe801-ovnkube-config\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.196829 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.196672 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-host-run-k8s-cni-cncf-io\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.196829 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.196707 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1b49c337-0256-42b7-9c89-1ceea7cbe801-env-overrides\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.196829 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.196735 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b3e07c44-d299-4e8d-b240-0988e28e9c2b-run\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.196829 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.196748 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/85d4dd2e-a53d-455e-a542-9ce5ebd5b479-cnibin\") pod \"multus-additional-cni-plugins-5g4kk\" (UID: \"85d4dd2e-a53d-455e-a542-9ce5ebd5b479\") " pod="openshift-multus/multus-additional-cni-plugins-5g4kk" Sep 09 02:10:43.196829 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.196784 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/85d4dd2e-a53d-455e-a542-9ce5ebd5b479-cnibin\") pod \"multus-additional-cni-plugins-5g4kk\" (UID: \"85d4dd2e-a53d-455e-a542-9ce5ebd5b479\") " pod="openshift-multus/multus-additional-cni-plugins-5g4kk" Sep 09 02:10:43.196829 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.196788 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-hostroot\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.196829 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.196819 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndzs7\" (UniqueName: \"kubernetes.io/projected/84536d51-1f06-4aac-ba98-7773a0e01692-kube-api-access-ndzs7\") pod \"node-ca-z7f6v\" (UID: \"84536d51-1f06-4aac-ba98-7773a0e01692\") " pod="openshift-image-registry/node-ca-z7f6v" Sep 09 02:10:43.196829 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.196835 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b3e07c44-d299-4e8d-b240-0988e28e9c2b-etc-sysconfig\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.196829 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.196847 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-host-kubelet\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.196829 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.196873 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-host-slash\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.196829 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.196878 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-hostroot\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.197909 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.196898 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bg7s8\" (UniqueName: \"kubernetes.io/projected/1b49c337-0256-42b7-9c89-1ceea7cbe801-kube-api-access-bg7s8\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.197909 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.196920 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-host-kubelet\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.197909 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.196925 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7aaa0ef6-52d4-4878-9a87-8cc0da967919-multus-daemon-config\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.197909 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.196951 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-etc-kubernetes\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.197909 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.196975 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b3e07c44-d299-4e8d-b240-0988e28e9c2b-host\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.197909 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.196994 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-node-log\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.197909 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197015 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-host-var-lib-kubelet\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.197909 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197038 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/84536d51-1f06-4aac-ba98-7773a0e01692-host\") pod \"node-ca-z7f6v\" (UID: \"84536d51-1f06-4aac-ba98-7773a0e01692\") " pod="openshift-image-registry/node-ca-z7f6v" Sep 09 02:10:43.197909 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197074 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3e07c44-d299-4e8d-b240-0988e28e9c2b-var-lib-kubelet\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.197909 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197093 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b3e07c44-d299-4e8d-b240-0988e28e9c2b-tmp\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.197909 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197104 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-host-run-ovn-kubernetes\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.197909 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197117 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-run-ovn\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.197909 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197152 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-run-ovn\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.197909 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197156 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/85d4dd2e-a53d-455e-a542-9ce5ebd5b479-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5g4kk\" (UID: \"85d4dd2e-a53d-455e-a542-9ce5ebd5b479\") " pod="openshift-multus/multus-additional-cni-plugins-5g4kk" Sep 09 02:10:43.197909 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197180 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-host-slash\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.197909 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197186 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-host-run-multus-certs\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.197909 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197216 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3e07c44-d299-4e8d-b240-0988e28e9c2b-etc-kubernetes\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.197909 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197290 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.198819 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197314 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1b49c337-0256-42b7-9c89-1ceea7cbe801-ovnkube-script-lib\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.198819 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197320 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/85d4dd2e-a53d-455e-a542-9ce5ebd5b479-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5g4kk\" (UID: \"85d4dd2e-a53d-455e-a542-9ce5ebd5b479\") " pod="openshift-multus/multus-additional-cni-plugins-5g4kk" Sep 09 02:10:43.198819 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197375 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-cnibin\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.198819 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197408 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7aaa0ef6-52d4-4878-9a87-8cc0da967919-cni-binary-copy\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.198819 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197410 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-host-var-lib-kubelet\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.198819 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197408 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b3e07c44-d299-4e8d-b240-0988e28e9c2b-host\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.198819 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197421 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-etc-kubernetes\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.198819 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197457 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.198819 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197477 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129-kubelet-config\") pod \"global-pull-secret-syncer-mzt99\" (UID: \"0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129\") " pod="kube-system/global-pull-secret-syncer-mzt99" Sep 09 02:10:43.198819 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197480 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3e07c44-d299-4e8d-b240-0988e28e9c2b-etc-kubernetes\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.198819 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197435 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129-kubelet-config\") pod \"global-pull-secret-syncer-mzt99\" (UID: \"0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129\") " pod="kube-system/global-pull-secret-syncer-mzt99" Sep 09 02:10:43.198819 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197514 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-node-log\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.198819 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197521 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-systemd-units\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.198819 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197480 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/84536d51-1f06-4aac-ba98-7773a0e01692-host\") pod \"node-ca-z7f6v\" (UID: \"84536d51-1f06-4aac-ba98-7773a0e01692\") " pod="openshift-image-registry/node-ca-z7f6v" Sep 09 02:10:43.198819 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197531 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3e07c44-d299-4e8d-b240-0988e28e9c2b-var-lib-kubelet\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.198819 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197549 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-var-lib-openvswitch\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.198819 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197568 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-cnibin\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.198819 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197572 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-systemd-units\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.199807 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197576 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129-original-pull-secret\") pod \"global-pull-secret-syncer-mzt99\" (UID: \"0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129\") " pod="kube-system/global-pull-secret-syncer-mzt99" Sep 09 02:10:43.199807 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197647 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b3e07c44-d299-4e8d-b240-0988e28e9c2b-etc-sysctl-conf\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.199807 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:43.197659 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 02:10:43.199807 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197680 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-var-lib-openvswitch\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.199807 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197714 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/85d4dd2e-a53d-455e-a542-9ce5ebd5b479-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5g4kk\" (UID: \"85d4dd2e-a53d-455e-a542-9ce5ebd5b479\") " pod="openshift-multus/multus-additional-cni-plugins-5g4kk" Sep 09 02:10:43.199807 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197736 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b3e07c44-d299-4e8d-b240-0988e28e9c2b-sys\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.199807 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:43.197740 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129-original-pull-secret podName:0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129 nodeName:}" failed. No retries permitted until 2026-09-09 02:10:43.697719555 +0000 UTC m=+3.072750949 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129-original-pull-secret") pod "global-pull-secret-syncer-mzt99" (UID: "0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129") : object "kube-system"/"original-pull-secret" not registered Sep 09 02:10:43.199807 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197686 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b3e07c44-d299-4e8d-b240-0988e28e9c2b-sys\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.199807 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197776 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-host-run-multus-certs\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.199807 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197796 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b3e07c44-d299-4e8d-b240-0988e28e9c2b-lib-modules\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.199807 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197814 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8dzv\" (UniqueName: \"kubernetes.io/projected/85d4dd2e-a53d-455e-a542-9ce5ebd5b479-kube-api-access-z8dzv\") pod \"multus-additional-cni-plugins-5g4kk\" (UID: \"85d4dd2e-a53d-455e-a542-9ce5ebd5b479\") " pod="openshift-multus/multus-additional-cni-plugins-5g4kk" Sep 09 02:10:43.199807 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197836 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-system-cni-dir\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.199807 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197861 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-host-var-lib-cni-bin\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.199807 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197900 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-system-cni-dir\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.199807 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197944 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/84536d51-1f06-4aac-ba98-7773a0e01692-serviceca\") pod \"node-ca-z7f6v\" (UID: \"84536d51-1f06-4aac-ba98-7773a0e01692\") " pod="openshift-image-registry/node-ca-z7f6v" Sep 09 02:10:43.199807 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197980 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpsfv\" (UniqueName: \"kubernetes.io/projected/b3e07c44-d299-4e8d-b240-0988e28e9c2b-kube-api-access-wpsfv\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.199807 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.197987 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b3e07c44-d299-4e8d-b240-0988e28e9c2b-lib-modules\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.200797 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198012 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-run-systemd\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.200797 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198049 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/85d4dd2e-a53d-455e-a542-9ce5ebd5b479-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5g4kk\" (UID: \"85d4dd2e-a53d-455e-a542-9ce5ebd5b479\") " pod="openshift-multus/multus-additional-cni-plugins-5g4kk" Sep 09 02:10:43.200797 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198070 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-run-systemd\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.200797 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198132 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-log-socket\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.200797 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198173 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-multus-cni-dir\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.200797 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198184 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-host-var-lib-cni-bin\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.200797 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198201 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a06a9649-a313-401a-80cb-c71fddc9d349-metrics-certs\") pod \"network-metrics-daemon-8x9lh\" (UID: \"a06a9649-a313-401a-80cb-c71fddc9d349\") " pod="openshift-multus/network-metrics-daemon-8x9lh" Sep 09 02:10:43.200797 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198226 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7aaa0ef6-52d4-4878-9a87-8cc0da967919-cni-binary-copy\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.200797 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198234 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-log-socket\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.200797 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198229 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-host-cni-bin\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.200797 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198275 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-host-cni-netd\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.200797 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198288 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-multus-cni-dir\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.200797 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198300 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/85d4dd2e-a53d-455e-a542-9ce5ebd5b479-system-cni-dir\") pod \"multus-additional-cni-plugins-5g4kk\" (UID: \"85d4dd2e-a53d-455e-a542-9ce5ebd5b479\") " pod="openshift-multus/multus-additional-cni-plugins-5g4kk" Sep 09 02:10:43.200797 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:43.198314 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 02:10:43.200797 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198327 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b3e07c44-d299-4e8d-b240-0988e28e9c2b-etc-modprobe-d\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.200797 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198335 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-host-cni-bin\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.200797 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198353 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b3e07c44-d299-4e8d-b240-0988e28e9c2b-etc-sysctl-d\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.200797 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198371 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-host-cni-netd\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.202628 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:43.198385 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a06a9649-a313-401a-80cb-c71fddc9d349-metrics-certs podName:a06a9649-a313-401a-80cb-c71fddc9d349 nodeName:}" failed. No retries permitted until 2026-09-09 02:10:43.698366925 +0000 UTC m=+3.073398346 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a06a9649-a313-401a-80cb-c71fddc9d349-metrics-certs") pod "network-metrics-daemon-8x9lh" (UID: "a06a9649-a313-401a-80cb-c71fddc9d349") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 02:10:43.202628 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198382 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/85d4dd2e-a53d-455e-a542-9ce5ebd5b479-system-cni-dir\") pod \"multus-additional-cni-plugins-5g4kk\" (UID: \"85d4dd2e-a53d-455e-a542-9ce5ebd5b479\") " pod="openshift-multus/multus-additional-cni-plugins-5g4kk" Sep 09 02:10:43.202628 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198226 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b3e07c44-d299-4e8d-b240-0988e28e9c2b-etc-sysctl-conf\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.202628 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198412 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7aaa0ef6-52d4-4878-9a87-8cc0da967919-multus-daemon-config\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.202628 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198406 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdhk5\" (UniqueName: \"kubernetes.io/projected/a06a9649-a313-401a-80cb-c71fddc9d349-kube-api-access-jdhk5\") pod \"network-metrics-daemon-8x9lh\" (UID: \"a06a9649-a313-401a-80cb-c71fddc9d349\") " pod="openshift-multus/network-metrics-daemon-8x9lh" Sep 09 02:10:43.202628 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198464 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/84536d51-1f06-4aac-ba98-7773a0e01692-serviceca\") pod \"node-ca-z7f6v\" (UID: \"84536d51-1f06-4aac-ba98-7773a0e01692\") " pod="openshift-image-registry/node-ca-z7f6v" Sep 09 02:10:43.202628 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198470 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b3e07c44-d299-4e8d-b240-0988e28e9c2b-etc-sysctl-d\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.202628 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198467 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b3e07c44-d299-4e8d-b240-0988e28e9c2b-etc-modprobe-d\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.202628 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198466 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-run-openvswitch\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.202628 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198501 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-run-openvswitch\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.202628 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198516 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/85d4dd2e-a53d-455e-a542-9ce5ebd5b479-cni-binary-copy\") pod \"multus-additional-cni-plugins-5g4kk\" (UID: \"85d4dd2e-a53d-455e-a542-9ce5ebd5b479\") " pod="openshift-multus/multus-additional-cni-plugins-5g4kk" Sep 09 02:10:43.202628 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198541 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b3e07c44-d299-4e8d-b240-0988e28e9c2b-etc-systemd\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.202628 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198581 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-etc-openvswitch\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.202628 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198609 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1b49c337-0256-42b7-9c89-1ceea7cbe801-ovn-node-metrics-cert\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.202628 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198639 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-multus-socket-dir-parent\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.202628 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198641 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b3e07c44-d299-4e8d-b240-0988e28e9c2b-etc-systemd\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.202628 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198666 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqgkn\" (UniqueName: \"kubernetes.io/projected/7aaa0ef6-52d4-4878-9a87-8cc0da967919-kube-api-access-kqgkn\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.203497 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198676 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-etc-openvswitch\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.203497 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198690 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129-dbus\") pod \"global-pull-secret-syncer-mzt99\" (UID: \"0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129\") " pod="kube-system/global-pull-secret-syncer-mzt99" Sep 09 02:10:43.203497 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198717 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-host-run-netns\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.203497 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198746 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/85d4dd2e-a53d-455e-a542-9ce5ebd5b479-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5g4kk\" (UID: \"85d4dd2e-a53d-455e-a542-9ce5ebd5b479\") " pod="openshift-multus/multus-additional-cni-plugins-5g4kk" Sep 09 02:10:43.203497 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198768 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1b49c337-0256-42b7-9c89-1ceea7cbe801-env-overrides\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.203497 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198775 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-host-var-lib-cni-multus\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.203497 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198803 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b3e07c44-d299-4e8d-b240-0988e28e9c2b-etc-tuned\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.203497 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198833 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-host-run-netns\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.203497 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198854 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1b49c337-0256-42b7-9c89-1ceea7cbe801-host-run-netns\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.203497 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198835 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129-dbus\") pod \"global-pull-secret-syncer-mzt99\" (UID: \"0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129\") " pod="kube-system/global-pull-secret-syncer-mzt99" Sep 09 02:10:43.203497 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198874 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-multus-socket-dir-parent\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.203497 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.198960 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1b49c337-0256-42b7-9c89-1ceea7cbe801-ovnkube-config\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.203497 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.199084 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-host-run-netns\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.203497 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.199123 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7aaa0ef6-52d4-4878-9a87-8cc0da967919-host-var-lib-cni-multus\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.203497 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.199268 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/85d4dd2e-a53d-455e-a542-9ce5ebd5b479-cni-binary-copy\") pod \"multus-additional-cni-plugins-5g4kk\" (UID: \"85d4dd2e-a53d-455e-a542-9ce5ebd5b479\") " pod="openshift-multus/multus-additional-cni-plugins-5g4kk" Sep 09 02:10:43.203497 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.199365 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/85d4dd2e-a53d-455e-a542-9ce5ebd5b479-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5g4kk\" (UID: \"85d4dd2e-a53d-455e-a542-9ce5ebd5b479\") " pod="openshift-multus/multus-additional-cni-plugins-5g4kk" Sep 09 02:10:43.203497 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.201856 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1b49c337-0256-42b7-9c89-1ceea7cbe801-ovn-node-metrics-cert\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.204343 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.202236 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b3e07c44-d299-4e8d-b240-0988e28e9c2b-tmp\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.204343 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.202453 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b3e07c44-d299-4e8d-b240-0988e28e9c2b-etc-tuned\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.207043 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.207014 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8dzv\" (UniqueName: \"kubernetes.io/projected/85d4dd2e-a53d-455e-a542-9ce5ebd5b479-kube-api-access-z8dzv\") pod \"multus-additional-cni-plugins-5g4kk\" (UID: \"85d4dd2e-a53d-455e-a542-9ce5ebd5b479\") " pod="openshift-multus/multus-additional-cni-plugins-5g4kk" Sep 09 02:10:43.208363 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.208315 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndzs7\" (UniqueName: \"kubernetes.io/projected/84536d51-1f06-4aac-ba98-7773a0e01692-kube-api-access-ndzs7\") pod \"node-ca-z7f6v\" (UID: \"84536d51-1f06-4aac-ba98-7773a0e01692\") " pod="openshift-image-registry/node-ca-z7f6v" Sep 09 02:10:43.209069 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.209043 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqgkn\" (UniqueName: \"kubernetes.io/projected/7aaa0ef6-52d4-4878-9a87-8cc0da967919-kube-api-access-kqgkn\") pod \"multus-psv7l\" (UID: \"7aaa0ef6-52d4-4878-9a87-8cc0da967919\") " pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.209659 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.209636 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpsfv\" (UniqueName: \"kubernetes.io/projected/b3e07c44-d299-4e8d-b240-0988e28e9c2b-kube-api-access-wpsfv\") pod \"tuned-rffj8\" (UID: \"b3e07c44-d299-4e8d-b240-0988e28e9c2b\") " pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.210254 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.210189 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdhk5\" (UniqueName: \"kubernetes.io/projected/a06a9649-a313-401a-80cb-c71fddc9d349-kube-api-access-jdhk5\") pod \"network-metrics-daemon-8x9lh\" (UID: \"a06a9649-a313-401a-80cb-c71fddc9d349\") " pod="openshift-multus/network-metrics-daemon-8x9lh" Sep 09 02:10:43.210578 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.210558 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bg7s8\" (UniqueName: \"kubernetes.io/projected/1b49c337-0256-42b7-9c89-1ceea7cbe801-kube-api-access-bg7s8\") pod \"ovnkube-node-9cxcz\" (UID: \"1b49c337-0256-42b7-9c89-1ceea7cbe801\") " pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.273142 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.273053 2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 02:10:43.290389 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.290362 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tvbf" Sep 09 02:10:43.300547 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.300520 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g8jhc" Sep 09 02:10:43.314394 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.314367 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wvtrf" Sep 09 02:10:43.320255 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.320230 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dt57p" Sep 09 02:10:43.328977 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.328948 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rffj8" Sep 09 02:10:43.338944 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.338910 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-psv7l" Sep 09 02:10:43.346935 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.346894 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-z7f6v" Sep 09 02:10:43.354787 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.354768 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:10:43.363726 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.363704 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5g4kk" Sep 09 02:10:43.702452 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.702357 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129-original-pull-secret\") pod \"global-pull-secret-syncer-mzt99\" (UID: \"0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129\") " pod="kube-system/global-pull-secret-syncer-mzt99" Sep 09 02:10:43.702452 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.702413 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a06a9649-a313-401a-80cb-c71fddc9d349-metrics-certs\") pod \"network-metrics-daemon-8x9lh\" (UID: \"a06a9649-a313-401a-80cb-c71fddc9d349\") " pod="openshift-multus/network-metrics-daemon-8x9lh" Sep 09 02:10:43.702452 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:43.702439 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4456s\" (UniqueName: \"kubernetes.io/projected/ae5099dc-0b2e-4a01-887c-3076a88ecbd8-kube-api-access-4456s\") pod \"network-check-target-4mzjj\" (UID: \"ae5099dc-0b2e-4a01-887c-3076a88ecbd8\") " pod="openshift-network-diagnostics/network-check-target-4mzjj" Sep 09 02:10:43.702696 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:43.702571 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 02:10:43.702696 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:43.702590 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 02:10:43.702696 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:43.702612 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 02:10:43.702696 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:43.702626 2561 projected.go:196] Error preparing data for projected volume kube-api-access-4456s for pod openshift-network-diagnostics/network-check-target-4mzjj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 02:10:43.702696 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:43.702573 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 02:10:43.702696 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:43.702669 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129-original-pull-secret podName:0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129 nodeName:}" failed. No retries permitted until 2026-09-09 02:10:44.702643226 +0000 UTC m=+4.077674759 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129-original-pull-secret") pod "global-pull-secret-syncer-mzt99" (UID: "0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129") : object "kube-system"/"original-pull-secret" not registered Sep 09 02:10:43.702696 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:43.702692 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae5099dc-0b2e-4a01-887c-3076a88ecbd8-kube-api-access-4456s podName:ae5099dc-0b2e-4a01-887c-3076a88ecbd8 nodeName:}" failed. No retries permitted until 2026-09-09 02:10:44.702681461 +0000 UTC m=+4.077712856 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4456s" (UniqueName: "kubernetes.io/projected/ae5099dc-0b2e-4a01-887c-3076a88ecbd8-kube-api-access-4456s") pod "network-check-target-4mzjj" (UID: "ae5099dc-0b2e-4a01-887c-3076a88ecbd8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 02:10:43.703054 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:43.702711 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a06a9649-a313-401a-80cb-c71fddc9d349-metrics-certs podName:a06a9649-a313-401a-80cb-c71fddc9d349 nodeName:}" failed. No retries permitted until 2026-09-09 02:10:44.702701452 +0000 UTC m=+4.077732843 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a06a9649-a313-401a-80cb-c71fddc9d349-metrics-certs") pod "network-metrics-daemon-8x9lh" (UID: "a06a9649-a313-401a-80cb-c71fddc9d349") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 02:10:44.086760 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:44.086609 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 02:05:43 +0000 UTC" deadline="2028-03-19 08:40:55.527366265 +0000 UTC" Sep 09 02:10:44.086760 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:44.086677 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13374h30m11.440693725s" Sep 09 02:10:44.105126 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:44.105087 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4mzjj" Sep 09 02:10:44.105294 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:44.105229 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4mzjj" podUID="ae5099dc-0b2e-4a01-887c-3076a88ecbd8" Sep 09 02:10:44.220399 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:44.220175 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84536d51_1f06_4aac_ba98_7773a0e01692.slice/crio-8aca8e77b21e3d5fea19848d3e1d3d637a9a1a9aa7b61e24b71bbb55baee2759 WatchSource:0}: Error finding container 8aca8e77b21e3d5fea19848d3e1d3d637a9a1a9aa7b61e24b71bbb55baee2759: Status 404 returned error can't find the container with id 8aca8e77b21e3d5fea19848d3e1d3d637a9a1a9aa7b61e24b71bbb55baee2759 Sep 09 02:10:44.221869 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:44.221849 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3e07c44_d299_4e8d_b240_0988e28e9c2b.slice/crio-7517c94eea2fc78e2bdd6caa07e51aca7b884a6cb36f30cc70bec58ff982afca WatchSource:0}: Error finding container 7517c94eea2fc78e2bdd6caa07e51aca7b884a6cb36f30cc70bec58ff982afca: Status 404 returned error can't find the container with id 7517c94eea2fc78e2bdd6caa07e51aca7b884a6cb36f30cc70bec58ff982afca Sep 09 02:10:44.225508 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:44.225485 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4974aeea_5424_46f6_b987_2eea8d42f810.slice/crio-a2128677a9ffd393df6b5b1ac7dcdc7ca36dc35a6f300dbe01addc41591767c2 WatchSource:0}: Error finding container a2128677a9ffd393df6b5b1ac7dcdc7ca36dc35a6f300dbe01addc41591767c2: Status 404 returned error can't find the container with id a2128677a9ffd393df6b5b1ac7dcdc7ca36dc35a6f300dbe01addc41591767c2 Sep 09 02:10:44.226251 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:44.226230 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09485ab0_cebd_48a9_ad8a_5e7a82cfb7ea.slice/crio-6c09da7db8f9f8538d5144e78fc3d95f5b4edfd64c5a4197d7b88f0b5dc297b7 WatchSource:0}: Error finding container 6c09da7db8f9f8538d5144e78fc3d95f5b4edfd64c5a4197d7b88f0b5dc297b7: Status 404 returned error can't find the container with id 6c09da7db8f9f8538d5144e78fc3d95f5b4edfd64c5a4197d7b88f0b5dc297b7 Sep 09 02:10:44.231034 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:44.231011 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79f5d659_762b_4f17_be10_469a57ff032b.slice/crio-91699260af9dced9788c2878a0ea1534c77261a4644296edf246b0bb685b4315 WatchSource:0}: Error finding container 91699260af9dced9788c2878a0ea1534c77261a4644296edf246b0bb685b4315: Status 404 returned error can't find the container with id 91699260af9dced9788c2878a0ea1534c77261a4644296edf246b0bb685b4315 Sep 09 02:10:44.231868 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:44.231832 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7aaa0ef6_52d4_4878_9a87_8cc0da967919.slice/crio-8f0d468256c95a7386508a25cce9829c9f41cdbe58288d7179557474566569c6 WatchSource:0}: Error finding container 8f0d468256c95a7386508a25cce9829c9f41cdbe58288d7179557474566569c6: Status 404 returned error can't find the container with id 8f0d468256c95a7386508a25cce9829c9f41cdbe58288d7179557474566569c6 Sep 09 02:10:44.232959 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:44.232862 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55dc8777_be96_4d4f_9f9f_8180a1e158fe.slice/crio-327a67be63aaeafba573269e28159f9b199d217982f309a070594ad97e495bb5 WatchSource:0}: Error finding container 327a67be63aaeafba573269e28159f9b199d217982f309a070594ad97e495bb5: Status 404 returned error can't find the container with id 327a67be63aaeafba573269e28159f9b199d217982f309a070594ad97e495bb5 Sep 09 02:10:44.233974 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:44.233947 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85d4dd2e_a53d_455e_a542_9ce5ebd5b479.slice/crio-8d500b29b90e0b6adc1419a6ec90ef6c85cf04af65c7ada07600e09b8cc97ab9 WatchSource:0}: Error finding container 8d500b29b90e0b6adc1419a6ec90ef6c85cf04af65c7ada07600e09b8cc97ab9: Status 404 returned error can't find the container with id 8d500b29b90e0b6adc1419a6ec90ef6c85cf04af65c7ada07600e09b8cc97ab9 Sep 09 02:10:44.236526 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:10:44.236505 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b49c337_0256_42b7_9c89_1ceea7cbe801.slice/crio-9438ac2e1c80714ea5ed5d7139e9077a04d6ba2aa2b45e93af37fc080f5e6edc WatchSource:0}: Error finding container 9438ac2e1c80714ea5ed5d7139e9077a04d6ba2aa2b45e93af37fc080f5e6edc: Status 404 returned error can't find the container with id 9438ac2e1c80714ea5ed5d7139e9077a04d6ba2aa2b45e93af37fc080f5e6edc Sep 09 02:10:44.709309 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:44.709266 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129-original-pull-secret\") pod \"global-pull-secret-syncer-mzt99\" (UID: \"0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129\") " pod="kube-system/global-pull-secret-syncer-mzt99" Sep 09 02:10:44.709505 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:44.709332 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a06a9649-a313-401a-80cb-c71fddc9d349-metrics-certs\") pod \"network-metrics-daemon-8x9lh\" (UID: \"a06a9649-a313-401a-80cb-c71fddc9d349\") " pod="openshift-multus/network-metrics-daemon-8x9lh" Sep 09 02:10:44.709505 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:44.709367 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4456s\" (UniqueName: \"kubernetes.io/projected/ae5099dc-0b2e-4a01-887c-3076a88ecbd8-kube-api-access-4456s\") pod \"network-check-target-4mzjj\" (UID: \"ae5099dc-0b2e-4a01-887c-3076a88ecbd8\") " pod="openshift-network-diagnostics/network-check-target-4mzjj" Sep 09 02:10:44.709505 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:44.709458 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 02:10:44.709505 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:44.709495 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 02:10:44.709659 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:44.709513 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 02:10:44.709659 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:44.709523 2561 projected.go:196] Error preparing data for projected volume kube-api-access-4456s for pod openshift-network-diagnostics/network-check-target-4mzjj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 02:10:44.709659 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:44.709538 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129-original-pull-secret podName:0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129 nodeName:}" failed. No retries permitted until 2026-09-09 02:10:46.709516137 +0000 UTC m=+6.084547541 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129-original-pull-secret") pod "global-pull-secret-syncer-mzt99" (UID: "0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129") : object "kube-system"/"original-pull-secret" not registered Sep 09 02:10:44.709659 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:44.709570 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae5099dc-0b2e-4a01-887c-3076a88ecbd8-kube-api-access-4456s podName:ae5099dc-0b2e-4a01-887c-3076a88ecbd8 nodeName:}" failed. No retries permitted until 2026-09-09 02:10:46.709552076 +0000 UTC m=+6.084583468 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4456s" (UniqueName: "kubernetes.io/projected/ae5099dc-0b2e-4a01-887c-3076a88ecbd8-kube-api-access-4456s") pod "network-check-target-4mzjj" (UID: "ae5099dc-0b2e-4a01-887c-3076a88ecbd8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 02:10:44.709659 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:44.709652 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 02:10:44.709837 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:44.709703 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a06a9649-a313-401a-80cb-c71fddc9d349-metrics-certs podName:a06a9649-a313-401a-80cb-c71fddc9d349 nodeName:}" failed. No retries permitted until 2026-09-09 02:10:46.709690665 +0000 UTC m=+6.084722067 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a06a9649-a313-401a-80cb-c71fddc9d349-metrics-certs") pod "network-metrics-daemon-8x9lh" (UID: "a06a9649-a313-401a-80cb-c71fddc9d349") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 02:10:45.087665 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:45.087584 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 02:05:43 +0000 UTC" deadline="2028-06-05 00:50:05.861966122 +0000 UTC" Sep 09 02:10:45.087665 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:45.087664 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15238h39m20.774307744s" Sep 09 02:10:45.107351 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:45.107322 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mzt99" Sep 09 02:10:45.107521 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:45.107455 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mzt99" podUID="0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129" Sep 09 02:10:45.107905 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:45.107881 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8x9lh" Sep 09 02:10:45.108021 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:45.107997 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8x9lh" podUID="a06a9649-a313-401a-80cb-c71fddc9d349" Sep 09 02:10:45.133704 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:45.133665 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wvtrf" event={"ID":"55dc8777-be96-4d4f-9f9f-8180a1e158fe","Type":"ContainerStarted","Data":"327a67be63aaeafba573269e28159f9b199d217982f309a070594ad97e495bb5"} Sep 09 02:10:45.141172 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:45.141131 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dt57p" event={"ID":"4974aeea-5424-46f6-b987-2eea8d42f810","Type":"ContainerStarted","Data":"a2128677a9ffd393df6b5b1ac7dcdc7ca36dc35a6f300dbe01addc41591767c2"} Sep 09 02:10:45.148290 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:45.148254 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tvbf" event={"ID":"09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea","Type":"ContainerStarted","Data":"6c09da7db8f9f8538d5144e78fc3d95f5b4edfd64c5a4197d7b88f0b5dc297b7"} Sep 09 02:10:45.154787 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:45.154751 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5g4kk" event={"ID":"85d4dd2e-a53d-455e-a542-9ce5ebd5b479","Type":"ContainerStarted","Data":"8d500b29b90e0b6adc1419a6ec90ef6c85cf04af65c7ada07600e09b8cc97ab9"} Sep 09 02:10:45.177001 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:45.176943 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-psv7l" event={"ID":"7aaa0ef6-52d4-4878-9a87-8cc0da967919","Type":"ContainerStarted","Data":"8f0d468256c95a7386508a25cce9829c9f41cdbe58288d7179557474566569c6"} Sep 09 02:10:45.184466 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:45.184433 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g8jhc" event={"ID":"79f5d659-762b-4f17-be10-469a57ff032b","Type":"ContainerStarted","Data":"91699260af9dced9788c2878a0ea1534c77261a4644296edf246b0bb685b4315"} Sep 09 02:10:45.187642 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:45.187569 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rffj8" event={"ID":"b3e07c44-d299-4e8d-b240-0988e28e9c2b","Type":"ContainerStarted","Data":"7517c94eea2fc78e2bdd6caa07e51aca7b884a6cb36f30cc70bec58ff982afca"} Sep 09 02:10:45.198121 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:45.198077 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-z7f6v" event={"ID":"84536d51-1f06-4aac-ba98-7773a0e01692","Type":"ContainerStarted","Data":"8aca8e77b21e3d5fea19848d3e1d3d637a9a1a9aa7b61e24b71bbb55baee2759"} Sep 09 02:10:45.208906 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:45.208818 2561 generic.go:356] "Generic (PLEG): container finished" podID="45b4e2b9f311e3b6046acb668fc1406e" containerID="a0510af4a3870f186e50696b3aa707cf4f365318d1c87bd702af1ca7ed80d927" exitCode=0 Sep 09 02:10:45.209458 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:45.209396 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-218.ec2.internal" event={"ID":"45b4e2b9f311e3b6046acb668fc1406e","Type":"ContainerDied","Data":"a0510af4a3870f186e50696b3aa707cf4f365318d1c87bd702af1ca7ed80d927"} Sep 09 02:10:45.223805 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:45.223769 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-218.ec2.internal" event={"ID":"fe8949b5a0140471a9c7b59c781130ce","Type":"ContainerStarted","Data":"1236a2404f8707b21ec4884ed832fef58f797656ece1d05b6d1bc84de8479580"} Sep 09 02:10:45.226045 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:45.226011 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" event={"ID":"1b49c337-0256-42b7-9c89-1ceea7cbe801","Type":"ContainerStarted","Data":"9438ac2e1c80714ea5ed5d7139e9077a04d6ba2aa2b45e93af37fc080f5e6edc"} Sep 09 02:10:45.250360 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:45.250294 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-218.ec2.internal" podStartSLOduration=2.250272771 podStartE2EDuration="2.250272771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 02:10:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:10:45.249751322 +0000 UTC m=+4.624782735" watchObservedRunningTime="2026-09-09 02:10:45.250272771 +0000 UTC m=+4.625304200" Sep 09 02:10:46.105617 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:46.105577 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4mzjj" Sep 09 02:10:46.106327 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:46.105715 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4mzjj" podUID="ae5099dc-0b2e-4a01-887c-3076a88ecbd8" Sep 09 02:10:46.246165 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:46.246060 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-218.ec2.internal" event={"ID":"45b4e2b9f311e3b6046acb668fc1406e","Type":"ContainerStarted","Data":"65ac5ac0df4e6472505d1399036777a69177ce77dd5f7a7468d40fd390bc1e97"} Sep 09 02:10:46.730486 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:46.729412 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129-original-pull-secret\") pod \"global-pull-secret-syncer-mzt99\" (UID: \"0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129\") " pod="kube-system/global-pull-secret-syncer-mzt99" Sep 09 02:10:46.730486 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:46.729478 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a06a9649-a313-401a-80cb-c71fddc9d349-metrics-certs\") pod \"network-metrics-daemon-8x9lh\" (UID: \"a06a9649-a313-401a-80cb-c71fddc9d349\") " pod="openshift-multus/network-metrics-daemon-8x9lh" Sep 09 02:10:46.730486 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:46.729514 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4456s\" (UniqueName: \"kubernetes.io/projected/ae5099dc-0b2e-4a01-887c-3076a88ecbd8-kube-api-access-4456s\") pod \"network-check-target-4mzjj\" (UID: \"ae5099dc-0b2e-4a01-887c-3076a88ecbd8\") " pod="openshift-network-diagnostics/network-check-target-4mzjj" Sep 09 02:10:46.730486 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:46.729667 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 02:10:46.730486 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:46.729686 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 02:10:46.730486 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:46.729698 2561 projected.go:196] Error preparing data for projected volume kube-api-access-4456s for pod openshift-network-diagnostics/network-check-target-4mzjj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 02:10:46.730486 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:46.729765 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae5099dc-0b2e-4a01-887c-3076a88ecbd8-kube-api-access-4456s podName:ae5099dc-0b2e-4a01-887c-3076a88ecbd8 nodeName:}" failed. No retries permitted until 2026-09-09 02:10:50.729745019 +0000 UTC m=+10.104776409 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4456s" (UniqueName: "kubernetes.io/projected/ae5099dc-0b2e-4a01-887c-3076a88ecbd8-kube-api-access-4456s") pod "network-check-target-4mzjj" (UID: "ae5099dc-0b2e-4a01-887c-3076a88ecbd8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 02:10:46.730486 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:46.730248 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 02:10:46.730486 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:46.730305 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129-original-pull-secret podName:0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129 nodeName:}" failed. No retries permitted until 2026-09-09 02:10:50.730286125 +0000 UTC m=+10.105317521 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129-original-pull-secret") pod "global-pull-secret-syncer-mzt99" (UID: "0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129") : object "kube-system"/"original-pull-secret" not registered Sep 09 02:10:46.730486 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:46.730370 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 02:10:46.730486 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:46.730401 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a06a9649-a313-401a-80cb-c71fddc9d349-metrics-certs podName:a06a9649-a313-401a-80cb-c71fddc9d349 nodeName:}" failed. No retries permitted until 2026-09-09 02:10:50.730390672 +0000 UTC m=+10.105422062 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a06a9649-a313-401a-80cb-c71fddc9d349-metrics-certs") pod "network-metrics-daemon-8x9lh" (UID: "a06a9649-a313-401a-80cb-c71fddc9d349") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 02:10:47.106304 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:47.106214 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8x9lh" Sep 09 02:10:47.106779 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:47.106367 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8x9lh" podUID="a06a9649-a313-401a-80cb-c71fddc9d349" Sep 09 02:10:47.107062 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:47.106872 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mzt99" Sep 09 02:10:47.107062 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:47.106983 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mzt99" podUID="0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129" Sep 09 02:10:48.105598 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:48.105553 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4mzjj" Sep 09 02:10:48.105812 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:48.105712 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4mzjj" podUID="ae5099dc-0b2e-4a01-887c-3076a88ecbd8" Sep 09 02:10:49.104919 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:49.104881 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mzt99" Sep 09 02:10:49.104919 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:49.104900 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8x9lh" Sep 09 02:10:49.105474 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:49.105049 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mzt99" podUID="0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129" Sep 09 02:10:49.105474 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:49.105181 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8x9lh" podUID="a06a9649-a313-401a-80cb-c71fddc9d349" Sep 09 02:10:50.104973 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:50.104889 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4mzjj" Sep 09 02:10:50.105463 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:50.105031 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4mzjj" podUID="ae5099dc-0b2e-4a01-887c-3076a88ecbd8" Sep 09 02:10:50.763205 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:50.763165 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a06a9649-a313-401a-80cb-c71fddc9d349-metrics-certs\") pod \"network-metrics-daemon-8x9lh\" (UID: \"a06a9649-a313-401a-80cb-c71fddc9d349\") " pod="openshift-multus/network-metrics-daemon-8x9lh" Sep 09 02:10:50.763366 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:50.763225 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4456s\" (UniqueName: \"kubernetes.io/projected/ae5099dc-0b2e-4a01-887c-3076a88ecbd8-kube-api-access-4456s\") pod \"network-check-target-4mzjj\" (UID: \"ae5099dc-0b2e-4a01-887c-3076a88ecbd8\") " pod="openshift-network-diagnostics/network-check-target-4mzjj" Sep 09 02:10:50.763366 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:50.763276 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129-original-pull-secret\") pod \"global-pull-secret-syncer-mzt99\" (UID: \"0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129\") " pod="kube-system/global-pull-secret-syncer-mzt99" Sep 09 02:10:50.763451 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:50.763393 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 02:10:50.763488 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:50.763460 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129-original-pull-secret podName:0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129 nodeName:}" failed. No retries permitted until 2026-09-09 02:10:58.763441691 +0000 UTC m=+18.138473122 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129-original-pull-secret") pod "global-pull-secret-syncer-mzt99" (UID: "0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129") : object "kube-system"/"original-pull-secret" not registered Sep 09 02:10:50.763856 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:50.763836 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 02:10:50.763927 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:50.763892 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a06a9649-a313-401a-80cb-c71fddc9d349-metrics-certs podName:a06a9649-a313-401a-80cb-c71fddc9d349 nodeName:}" failed. No retries permitted until 2026-09-09 02:10:58.763877617 +0000 UTC m=+18.138909005 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a06a9649-a313-401a-80cb-c71fddc9d349-metrics-certs") pod "network-metrics-daemon-8x9lh" (UID: "a06a9649-a313-401a-80cb-c71fddc9d349") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 02:10:50.763999 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:50.763957 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 02:10:50.763999 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:50.763972 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 02:10:50.763999 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:50.763981 2561 projected.go:196] Error preparing data for projected volume kube-api-access-4456s for pod openshift-network-diagnostics/network-check-target-4mzjj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 02:10:50.764236 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:50.764006 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae5099dc-0b2e-4a01-887c-3076a88ecbd8-kube-api-access-4456s podName:ae5099dc-0b2e-4a01-887c-3076a88ecbd8 nodeName:}" failed. No retries permitted until 2026-09-09 02:10:58.763999286 +0000 UTC m=+18.139030674 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4456s" (UniqueName: "kubernetes.io/projected/ae5099dc-0b2e-4a01-887c-3076a88ecbd8-kube-api-access-4456s") pod "network-check-target-4mzjj" (UID: "ae5099dc-0b2e-4a01-887c-3076a88ecbd8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 02:10:51.106696 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:51.106020 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8x9lh" Sep 09 02:10:51.106696 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:51.106176 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8x9lh" podUID="a06a9649-a313-401a-80cb-c71fddc9d349" Sep 09 02:10:51.106696 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:51.106516 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mzt99" Sep 09 02:10:51.106696 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:51.106607 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mzt99" podUID="0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129" Sep 09 02:10:52.105889 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:52.105158 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4mzjj" Sep 09 02:10:52.105889 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:52.105518 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4mzjj" podUID="ae5099dc-0b2e-4a01-887c-3076a88ecbd8" Sep 09 02:10:53.104777 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:53.104742 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mzt99" Sep 09 02:10:53.105275 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:53.104882 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mzt99" podUID="0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129" Sep 09 02:10:53.105335 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:53.105279 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8x9lh" Sep 09 02:10:53.105412 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:53.105384 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8x9lh" podUID="a06a9649-a313-401a-80cb-c71fddc9d349" Sep 09 02:10:54.105298 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:54.105255 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4mzjj" Sep 09 02:10:54.105684 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:54.105369 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4mzjj" podUID="ae5099dc-0b2e-4a01-887c-3076a88ecbd8" Sep 09 02:10:55.105446 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:55.105407 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8x9lh" Sep 09 02:10:55.105446 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:55.105407 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mzt99" Sep 09 02:10:55.105869 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:55.105532 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8x9lh" podUID="a06a9649-a313-401a-80cb-c71fddc9d349" Sep 09 02:10:55.105869 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:55.105640 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mzt99" podUID="0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129" Sep 09 02:10:56.105339 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:56.105305 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4mzjj" Sep 09 02:10:56.105512 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:56.105424 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4mzjj" podUID="ae5099dc-0b2e-4a01-887c-3076a88ecbd8" Sep 09 02:10:57.105129 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:57.105096 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mzt99" Sep 09 02:10:57.105322 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:57.105099 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8x9lh" Sep 09 02:10:57.105322 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:57.105236 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mzt99" podUID="0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129" Sep 09 02:10:57.105322 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:57.105298 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8x9lh" podUID="a06a9649-a313-401a-80cb-c71fddc9d349" Sep 09 02:10:58.104490 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:58.104457 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4mzjj" Sep 09 02:10:58.104939 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:58.104576 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4mzjj" podUID="ae5099dc-0b2e-4a01-887c-3076a88ecbd8" Sep 09 02:10:58.823882 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:58.823849 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129-original-pull-secret\") pod \"global-pull-secret-syncer-mzt99\" (UID: \"0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129\") " pod="kube-system/global-pull-secret-syncer-mzt99" Sep 09 02:10:58.823882 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:58.823887 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a06a9649-a313-401a-80cb-c71fddc9d349-metrics-certs\") pod \"network-metrics-daemon-8x9lh\" (UID: \"a06a9649-a313-401a-80cb-c71fddc9d349\") " pod="openshift-multus/network-metrics-daemon-8x9lh" Sep 09 02:10:58.824123 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:58.823906 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4456s\" (UniqueName: \"kubernetes.io/projected/ae5099dc-0b2e-4a01-887c-3076a88ecbd8-kube-api-access-4456s\") pod \"network-check-target-4mzjj\" (UID: \"ae5099dc-0b2e-4a01-887c-3076a88ecbd8\") " pod="openshift-network-diagnostics/network-check-target-4mzjj" Sep 09 02:10:58.824123 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:58.823987 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 02:10:58.824123 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:58.824004 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 02:10:58.824123 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:58.824006 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 02:10:58.824123 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:58.824058 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129-original-pull-secret podName:0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129 nodeName:}" failed. No retries permitted until 2026-09-09 02:11:14.824042265 +0000 UTC m=+34.199073652 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129-original-pull-secret") pod "global-pull-secret-syncer-mzt99" (UID: "0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129") : object "kube-system"/"original-pull-secret" not registered Sep 09 02:10:58.824123 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:58.824078 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a06a9649-a313-401a-80cb-c71fddc9d349-metrics-certs podName:a06a9649-a313-401a-80cb-c71fddc9d349 nodeName:}" failed. No retries permitted until 2026-09-09 02:11:14.824064542 +0000 UTC m=+34.199095934 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a06a9649-a313-401a-80cb-c71fddc9d349-metrics-certs") pod "network-metrics-daemon-8x9lh" (UID: "a06a9649-a313-401a-80cb-c71fddc9d349") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 02:10:58.824123 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:58.824029 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 02:10:58.824123 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:58.824094 2561 projected.go:196] Error preparing data for projected volume kube-api-access-4456s for pod openshift-network-diagnostics/network-check-target-4mzjj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 02:10:58.824123 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:58.824119 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae5099dc-0b2e-4a01-887c-3076a88ecbd8-kube-api-access-4456s podName:ae5099dc-0b2e-4a01-887c-3076a88ecbd8 nodeName:}" failed. No retries permitted until 2026-09-09 02:11:14.824113151 +0000 UTC m=+34.199144544 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4456s" (UniqueName: "kubernetes.io/projected/ae5099dc-0b2e-4a01-887c-3076a88ecbd8-kube-api-access-4456s") pod "network-check-target-4mzjj" (UID: "ae5099dc-0b2e-4a01-887c-3076a88ecbd8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 02:10:59.104750 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:59.104671 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mzt99" Sep 09 02:10:59.105158 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:59.104787 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mzt99" podUID="0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129" Sep 09 02:10:59.105158 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:10:59.104854 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8x9lh" Sep 09 02:10:59.105158 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:10:59.104982 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8x9lh" podUID="a06a9649-a313-401a-80cb-c71fddc9d349" Sep 09 02:11:00.105275 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:00.105242 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4mzjj" Sep 09 02:11:00.105659 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:00.105353 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4mzjj" podUID="ae5099dc-0b2e-4a01-887c-3076a88ecbd8" Sep 09 02:11:01.108345 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:01.108316 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mzt99" Sep 09 02:11:01.108815 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:01.108429 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mzt99" podUID="0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129" Sep 09 02:11:01.108815 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:01.108494 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8x9lh" Sep 09 02:11:01.108815 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:01.108596 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8x9lh" podUID="a06a9649-a313-401a-80cb-c71fddc9d349" Sep 09 02:11:02.105314 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:02.105284 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4mzjj" Sep 09 02:11:02.105494 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:02.105394 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4mzjj" podUID="ae5099dc-0b2e-4a01-887c-3076a88ecbd8" Sep 09 02:11:03.104930 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:03.104695 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mzt99" Sep 09 02:11:03.105455 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:03.104751 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8x9lh" Sep 09 02:11:03.105455 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:03.105059 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mzt99" podUID="0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129" Sep 09 02:11:03.105455 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:03.105095 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8x9lh" podUID="a06a9649-a313-401a-80cb-c71fddc9d349" Sep 09 02:11:03.288902 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:03.288856 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tvbf" event={"ID":"09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea","Type":"ContainerStarted","Data":"99741a298389faecfb3ef4c3b7f777cb7d5bc0192ecd18c23a6e7c5ec9f6e601"} Sep 09 02:11:03.290796 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:03.290768 2561 generic.go:356] "Generic (PLEG): container finished" podID="85d4dd2e-a53d-455e-a542-9ce5ebd5b479" containerID="23e975fd9a34a7d00fd37c034f585f1eedab30eccf65574b2174a59582e578cb" exitCode=0 Sep 09 02:11:03.290941 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:03.290836 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5g4kk" event={"ID":"85d4dd2e-a53d-455e-a542-9ce5ebd5b479","Type":"ContainerDied","Data":"23e975fd9a34a7d00fd37c034f585f1eedab30eccf65574b2174a59582e578cb"} Sep 09 02:11:03.296840 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:03.296808 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-psv7l" event={"ID":"7aaa0ef6-52d4-4878-9a87-8cc0da967919","Type":"ContainerStarted","Data":"583ecfa355fbad2bd90e7050184d9e7313f70403cda4eaf8141a21cc24d7c2e5"} Sep 09 02:11:03.299033 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:03.298989 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g8jhc" event={"ID":"79f5d659-762b-4f17-be10-469a57ff032b","Type":"ContainerStarted","Data":"bda9e82dc931b00bb3937f41ced241ea5cba81c87c61000ed8951cbc5f901f5b"} Sep 09 02:11:03.301227 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:03.301202 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rffj8" event={"ID":"b3e07c44-d299-4e8d-b240-0988e28e9c2b","Type":"ContainerStarted","Data":"0659dd3d2a00a31f87dbe83e3284ab009ed9a4de64d310c62117e5f7d1d5463b"} Sep 09 02:11:03.302968 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:03.302942 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-z7f6v" event={"ID":"84536d51-1f06-4aac-ba98-7773a0e01692","Type":"ContainerStarted","Data":"996006bbf52a679fe1e1e8a4fca32c20559b6aa43f7490eeee5d34da3af0ee67"} Sep 09 02:11:03.306377 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:03.306354 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" event={"ID":"1b49c337-0256-42b7-9c89-1ceea7cbe801","Type":"ContainerStarted","Data":"2030ef4509fc66a879b77313b4e81644945e6756c3d9dc217c034a6d901b6cb4"} Sep 09 02:11:03.306478 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:03.306419 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" event={"ID":"1b49c337-0256-42b7-9c89-1ceea7cbe801","Type":"ContainerStarted","Data":"8c202370869f4c630e7118e9d7be6f937cc74a437cc3fec22d197506cf498d78"} Sep 09 02:11:03.306478 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:03.306449 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" event={"ID":"1b49c337-0256-42b7-9c89-1ceea7cbe801","Type":"ContainerStarted","Data":"526de8df416cb5492852c90e6ce6a3cbae82196fc0db42d9b3e602cda75ce7ca"} Sep 09 02:11:03.306478 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:03.306478 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" event={"ID":"1b49c337-0256-42b7-9c89-1ceea7cbe801","Type":"ContainerStarted","Data":"b60675d56df0a577ad4fe83de345adbad3afd9a7ffc9230128ffd237566b5f2f"} Sep 09 02:11:03.306643 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:03.306505 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" event={"ID":"1b49c337-0256-42b7-9c89-1ceea7cbe801","Type":"ContainerStarted","Data":"c3ebda268844da7282e65cf798898218698e2c9233618a435945781d5c992562"} Sep 09 02:11:03.306643 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:03.306539 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" event={"ID":"1b49c337-0256-42b7-9c89-1ceea7cbe801","Type":"ContainerStarted","Data":"2b7ce2f537d407af0d197c5f67bd6c317d5019ee7b222ff5149adef689aefc07"} Sep 09 02:11:03.308207 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:03.308183 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wvtrf" event={"ID":"55dc8777-be96-4d4f-9f9f-8180a1e158fe","Type":"ContainerStarted","Data":"964d388f8eec16ef3c2af22fd644452ac91861185eb63861f6787adac3200648"} Sep 09 02:11:03.310620 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:03.310581 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-218.ec2.internal" podStartSLOduration=20.310569867 podStartE2EDuration="20.310569867s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 02:10:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:10:46.264960906 +0000 UTC m=+5.639992320" watchObservedRunningTime="2026-09-09 02:11:03.310569867 +0000 UTC m=+22.685601276" Sep 09 02:11:03.325276 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:03.325187 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-wvtrf" podStartSLOduration=3.931363396 podStartE2EDuration="22.325173098s" totalImagesPullingTime="18.393809702s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 02:10:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:11:03.324669627 +0000 UTC m=+22.699701037" watchObservedRunningTime="2026-09-09 02:11:03.325173098 +0000 UTC m=+22.700204507" Sep 09 02:11:03.339386 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:03.339335 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-g8jhc" podStartSLOduration=7.891794347 podStartE2EDuration="22.339318663s" totalImagesPullingTime="14.447524316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:10:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:11:03.339253467 +0000 UTC m=+22.714284875" watchObservedRunningTime="2026-09-09 02:11:03.339318663 +0000 UTC m=+22.714350074" Sep 09 02:11:03.358681 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:03.358639 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-rffj8" podStartSLOduration=3.936155325 podStartE2EDuration="22.358626383s" totalImagesPullingTime="18.422471058s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 02:10:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:11:03.358407364 +0000 UTC m=+22.733438776" watchObservedRunningTime="2026-09-09 02:11:03.358626383 +0000 UTC m=+22.733657793" Sep 09 02:11:03.373933 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:03.373878 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-psv7l" podStartSLOduration=3.91757015 podStartE2EDuration="22.373855561s" totalImagesPullingTime="18.456285411s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 02:10:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:11:03.373395838 +0000 UTC m=+22.748427249" watchObservedRunningTime="2026-09-09 02:11:03.373855561 +0000 UTC m=+22.748886972" Sep 09 02:11:03.390033 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:03.389925 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-z7f6v" podStartSLOduration=3.988990235 podStartE2EDuration="22.389912195s" totalImagesPullingTime="18.40092196s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 02:10:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:11:03.389586032 +0000 UTC m=+22.764617443" watchObservedRunningTime="2026-09-09 02:11:03.389912195 +0000 UTC m=+22.764943604" Sep 09 02:11:04.090854 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:04.090598 2561 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 09 02:11:04.105110 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:04.105080 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4mzjj" Sep 09 02:11:04.105568 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:04.105194 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4mzjj" podUID="ae5099dc-0b2e-4a01-887c-3076a88ecbd8" Sep 09 02:11:04.312901 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:04.312773 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dt57p" event={"ID":"4974aeea-5424-46f6-b987-2eea8d42f810","Type":"ContainerStarted","Data":"ffba3dc3c678de5c144deeca352a1b6df7ccb9a396fc07b5b2a63698e7f10958"} Sep 09 02:11:04.315028 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:04.314821 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tvbf" event={"ID":"09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea","Type":"ContainerStarted","Data":"248c32609015b395fa7a21d10a78542e4a3201efaa57f8b93b1c66e12b0e411b"} Sep 09 02:11:04.328564 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:04.328519 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-dt57p" podStartSLOduration=4.93303919 podStartE2EDuration="23.328505431s" totalImagesPullingTime="18.395466241s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 02:10:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:11:04.328179925 +0000 UTC m=+23.703211332" watchObservedRunningTime="2026-09-09 02:11:04.328505431 +0000 UTC m=+23.703536836" Sep 09 02:11:05.049275 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:05.049098 2561 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-09T02:11:04.090834169Z","UUID":"fb69429c-8488-4d8e-aee8-b4798acd84b6","Handler":null,"Name":"","Endpoint":""} Sep 09 02:11:05.050825 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:05.050782 2561 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 09 02:11:05.050825 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:05.050819 2561 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 09 02:11:05.105607 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:05.105574 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mzt99" Sep 09 02:11:05.105607 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:05.105591 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8x9lh" Sep 09 02:11:05.106138 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:05.105702 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mzt99" podUID="0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129" Sep 09 02:11:05.106138 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:05.105814 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8x9lh" podUID="a06a9649-a313-401a-80cb-c71fddc9d349" Sep 09 02:11:06.105348 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:06.105312 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4mzjj" Sep 09 02:11:06.105520 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:06.105425 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4mzjj" podUID="ae5099dc-0b2e-4a01-887c-3076a88ecbd8" Sep 09 02:11:06.270720 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:06.270684 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-g8jhc" Sep 09 02:11:06.271274 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:06.271255 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-g8jhc" Sep 09 02:11:06.323991 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:06.323955 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" event={"ID":"1b49c337-0256-42b7-9c89-1ceea7cbe801","Type":"ContainerStarted","Data":"e047ff07ed9bca096055041c415acb00cebefafefec29db7342fd38c0b5da9ad"} Sep 09 02:11:06.326386 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:06.326357 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tvbf" event={"ID":"09485ab0-cebd-48a9-ad8a-5e7a82cfb7ea","Type":"ContainerStarted","Data":"387a025458d82512be6fec853bb3073e45c4008feb18ff14f5bb616fd967dd0e"} Sep 09 02:11:06.326793 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:06.326766 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-g8jhc" Sep 09 02:11:06.327101 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:06.327085 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-g8jhc" Sep 09 02:11:06.345392 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:06.345338 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tvbf" podStartSLOduration=4.353888009 podStartE2EDuration="25.345318917s" totalImagesPullingTime="20.991430908s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 02:10:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:11:06.345313072 +0000 UTC m=+25.720344483" watchObservedRunningTime="2026-09-09 02:11:06.345318917 +0000 UTC m=+25.720350330" Sep 09 02:11:07.104709 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:07.104673 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8x9lh" Sep 09 02:11:07.104709 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:07.104713 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mzt99" Sep 09 02:11:07.104986 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:07.104816 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8x9lh" podUID="a06a9649-a313-401a-80cb-c71fddc9d349" Sep 09 02:11:07.104986 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:07.104934 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mzt99" podUID="0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129" Sep 09 02:11:08.105677 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:08.105533 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4mzjj" Sep 09 02:11:08.105677 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:08.105642 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4mzjj" podUID="ae5099dc-0b2e-4a01-887c-3076a88ecbd8" Sep 09 02:11:08.334656 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:08.334485 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" event={"ID":"1b49c337-0256-42b7-9c89-1ceea7cbe801","Type":"ContainerStarted","Data":"8e495c31dcb90bd4ac4b3de7d87e9a55838b38313a205953313b245635b7b016"} Sep 09 02:11:08.361789 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:08.361735 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" podStartSLOduration=7.92047993 podStartE2EDuration="26.361720935s" totalImagesPullingTime="18.441241005s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 02:10:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:11:08.360792389 +0000 UTC m=+27.735823799" watchObservedRunningTime="2026-09-09 02:11:08.361720935 +0000 UTC m=+27.736752344" Sep 09 02:11:09.105335 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:09.105301 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mzt99" Sep 09 02:11:09.105622 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:09.105406 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mzt99" podUID="0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129" Sep 09 02:11:09.105622 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:09.105461 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8x9lh" Sep 09 02:11:09.105622 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:09.105555 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8x9lh" podUID="a06a9649-a313-401a-80cb-c71fddc9d349" Sep 09 02:11:09.339258 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:09.339222 2561 generic.go:356] "Generic (PLEG): container finished" podID="85d4dd2e-a53d-455e-a542-9ce5ebd5b479" containerID="eb429be596250340369e22132e0dcdea5b46949c4ce2a5f30b7dd1985193bcc3" exitCode=0 Sep 09 02:11:09.339431 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:09.339308 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5g4kk" event={"ID":"85d4dd2e-a53d-455e-a542-9ce5ebd5b479","Type":"ContainerDied","Data":"eb429be596250340369e22132e0dcdea5b46949c4ce2a5f30b7dd1985193bcc3"} Sep 09 02:11:09.339840 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:09.339815 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:11:09.339951 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:09.339893 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:11:09.339951 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:09.339943 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:11:09.361616 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:09.361344 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:11:09.362541 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:09.362035 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:11:10.002690 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:10.002456 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4mzjj"] Sep 09 02:11:10.002890 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:10.002806 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4mzjj" Sep 09 02:11:10.002940 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:10.002917 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4mzjj" podUID="ae5099dc-0b2e-4a01-887c-3076a88ecbd8" Sep 09 02:11:10.003152 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:10.003133 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mzt99"] Sep 09 02:11:10.003272 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:10.003260 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mzt99" Sep 09 02:11:10.003345 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:10.003328 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mzt99" podUID="0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129" Sep 09 02:11:10.004825 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:10.004751 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8x9lh"] Sep 09 02:11:10.004964 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:10.004865 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8x9lh" Sep 09 02:11:10.005034 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:10.004963 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8x9lh" podUID="a06a9649-a313-401a-80cb-c71fddc9d349" Sep 09 02:11:10.343927 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:10.343889 2561 generic.go:356] "Generic (PLEG): container finished" podID="85d4dd2e-a53d-455e-a542-9ce5ebd5b479" containerID="8e6ef0ee9c15e1ef75bfee19ce3803fef2ff164e409b5567c8e27e77f0ac17ec" exitCode=0 Sep 09 02:11:10.344463 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:10.343964 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5g4kk" event={"ID":"85d4dd2e-a53d-455e-a542-9ce5ebd5b479","Type":"ContainerDied","Data":"8e6ef0ee9c15e1ef75bfee19ce3803fef2ff164e409b5567c8e27e77f0ac17ec"} Sep 09 02:11:11.106344 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:11.106308 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mzt99" Sep 09 02:11:11.106515 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:11.106395 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mzt99" podUID="0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129" Sep 09 02:11:11.348280 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:11.348245 2561 generic.go:356] "Generic (PLEG): container finished" podID="85d4dd2e-a53d-455e-a542-9ce5ebd5b479" containerID="2bc0e1583f59f1526920a55dd284e6336b5d9ac6aecbb80e1c0440bfcade7b98" exitCode=0 Sep 09 02:11:11.348696 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:11.348334 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5g4kk" event={"ID":"85d4dd2e-a53d-455e-a542-9ce5ebd5b479","Type":"ContainerDied","Data":"2bc0e1583f59f1526920a55dd284e6336b5d9ac6aecbb80e1c0440bfcade7b98"} Sep 09 02:11:12.104723 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:12.104542 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8x9lh" Sep 09 02:11:12.104723 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:12.104676 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8x9lh" podUID="a06a9649-a313-401a-80cb-c71fddc9d349" Sep 09 02:11:12.104961 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:12.104755 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4mzjj" Sep 09 02:11:12.104961 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:12.104858 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4mzjj" podUID="ae5099dc-0b2e-4a01-887c-3076a88ecbd8" Sep 09 02:11:13.105426 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:13.105396 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mzt99" Sep 09 02:11:13.105879 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:13.105535 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mzt99" podUID="0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129" Sep 09 02:11:14.105298 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:14.105219 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8x9lh" Sep 09 02:11:14.105298 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:14.105248 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4mzjj" Sep 09 02:11:14.105502 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:14.105377 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8x9lh" podUID="a06a9649-a313-401a-80cb-c71fddc9d349" Sep 09 02:11:14.105823 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:14.105511 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4mzjj" podUID="ae5099dc-0b2e-4a01-887c-3076a88ecbd8" Sep 09 02:11:14.843225 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:14.843185 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129-original-pull-secret\") pod \"global-pull-secret-syncer-mzt99\" (UID: \"0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129\") " pod="kube-system/global-pull-secret-syncer-mzt99" Sep 09 02:11:14.843501 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:14.843245 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a06a9649-a313-401a-80cb-c71fddc9d349-metrics-certs\") pod \"network-metrics-daemon-8x9lh\" (UID: \"a06a9649-a313-401a-80cb-c71fddc9d349\") " pod="openshift-multus/network-metrics-daemon-8x9lh" Sep 09 02:11:14.843501 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:14.843280 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4456s\" (UniqueName: \"kubernetes.io/projected/ae5099dc-0b2e-4a01-887c-3076a88ecbd8-kube-api-access-4456s\") pod \"network-check-target-4mzjj\" (UID: \"ae5099dc-0b2e-4a01-887c-3076a88ecbd8\") " pod="openshift-network-diagnostics/network-check-target-4mzjj" Sep 09 02:11:14.843501 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:14.843348 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 02:11:14.843501 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:14.843409 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 02:11:14.843501 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:14.843430 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129-original-pull-secret podName:0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129 nodeName:}" failed. No retries permitted until 2026-09-09 02:11:46.843412249 +0000 UTC m=+66.218443638 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129-original-pull-secret") pod "global-pull-secret-syncer-mzt99" (UID: "0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129") : object "kube-system"/"original-pull-secret" not registered Sep 09 02:11:14.843501 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:14.843472 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a06a9649-a313-401a-80cb-c71fddc9d349-metrics-certs podName:a06a9649-a313-401a-80cb-c71fddc9d349 nodeName:}" failed. No retries permitted until 2026-09-09 02:11:46.843454865 +0000 UTC m=+66.218486254 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a06a9649-a313-401a-80cb-c71fddc9d349-metrics-certs") pod "network-metrics-daemon-8x9lh" (UID: "a06a9649-a313-401a-80cb-c71fddc9d349") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 02:11:14.843501 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:14.843412 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 02:11:14.843501 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:14.843506 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 02:11:14.843853 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:14.843524 2561 projected.go:196] Error preparing data for projected volume kube-api-access-4456s for pod openshift-network-diagnostics/network-check-target-4mzjj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 02:11:14.843853 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:14.843569 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae5099dc-0b2e-4a01-887c-3076a88ecbd8-kube-api-access-4456s podName:ae5099dc-0b2e-4a01-887c-3076a88ecbd8 nodeName:}" failed. No retries permitted until 2026-09-09 02:11:46.843557832 +0000 UTC m=+66.218589230 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4456s" (UniqueName: "kubernetes.io/projected/ae5099dc-0b2e-4a01-887c-3076a88ecbd8-kube-api-access-4456s") pod "network-check-target-4mzjj" (UID: "ae5099dc-0b2e-4a01-887c-3076a88ecbd8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 02:11:15.105003 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.104915 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mzt99" Sep 09 02:11:15.105183 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:15.105060 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mzt99" podUID="0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129" Sep 09 02:11:15.402121 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.402048 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-218.ec2.internal" event="NodeReady" Sep 09 02:11:15.402454 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.402191 2561 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 09 02:11:15.447268 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.447227 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vpmqs"] Sep 09 02:11:15.461352 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.461321 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gbk4t"] Sep 09 02:11:15.461489 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.461436 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vpmqs" Sep 09 02:11:15.464159 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.464130 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 09 02:11:15.464300 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.464252 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mh8vv\"" Sep 09 02:11:15.464498 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.464482 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 09 02:11:15.483855 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.483820 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vpmqs"] Sep 09 02:11:15.484160 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.483866 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gbk4t" Sep 09 02:11:15.484160 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.483875 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gbk4t"] Sep 09 02:11:15.484160 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.484113 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-75d99c484-c2kht"] Sep 09 02:11:15.486618 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.486600 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 09 02:11:15.486618 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.486614 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 09 02:11:15.486782 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.486647 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 09 02:11:15.487426 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.487409 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wrp7f\"" Sep 09 02:11:15.498822 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.498803 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-nk9tm"] Sep 09 02:11:15.498926 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.498911 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-c2kht" Sep 09 02:11:15.501355 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.501324 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Sep 09 02:11:15.501355 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.501324 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-k6c7t\"" Sep 09 02:11:15.501512 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.501385 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Sep 09 02:11:15.501512 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.501388 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Sep 09 02:11:15.501734 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.501719 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Sep 09 02:11:15.519219 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.519199 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-c2kht"] Sep 09 02:11:15.519303 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.519229 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nk9tm"] Sep 09 02:11:15.519340 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.519309 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nk9tm" Sep 09 02:11:15.524815 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.524788 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 09 02:11:15.525112 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.525092 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 09 02:11:15.525316 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.525297 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 09 02:11:15.525555 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.525537 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 09 02:11:15.525613 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.525541 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-c892g\"" Sep 09 02:11:15.548001 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.547977 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkjcg\" (UniqueName: \"kubernetes.io/projected/c3847832-aac1-4411-8629-57ac0ba9427b-kube-api-access-jkjcg\") pod \"ingress-canary-gbk4t\" (UID: \"c3847832-aac1-4411-8629-57ac0ba9427b\") " pod="openshift-ingress-canary/ingress-canary-gbk4t" Sep 09 02:11:15.548114 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.548006 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ec1c2f7-97f8-463a-be11-27db70b7f648-metrics-tls\") pod \"dns-default-vpmqs\" (UID: \"7ec1c2f7-97f8-463a-be11-27db70b7f648\") " pod="openshift-dns/dns-default-vpmqs" Sep 09 02:11:15.548114 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.548039 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7ec1c2f7-97f8-463a-be11-27db70b7f648-config-volume\") pod \"dns-default-vpmqs\" (UID: \"7ec1c2f7-97f8-463a-be11-27db70b7f648\") " pod="openshift-dns/dns-default-vpmqs" Sep 09 02:11:15.548114 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.548061 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ec1c2f7-97f8-463a-be11-27db70b7f648-tmp-dir\") pod \"dns-default-vpmqs\" (UID: \"7ec1c2f7-97f8-463a-be11-27db70b7f648\") " pod="openshift-dns/dns-default-vpmqs" Sep 09 02:11:15.548114 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.548093 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzdzt\" (UniqueName: \"kubernetes.io/projected/7ec1c2f7-97f8-463a-be11-27db70b7f648-kube-api-access-mzdzt\") pod \"dns-default-vpmqs\" (UID: \"7ec1c2f7-97f8-463a-be11-27db70b7f648\") " pod="openshift-dns/dns-default-vpmqs" Sep 09 02:11:15.548258 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.548122 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c3847832-aac1-4411-8629-57ac0ba9427b-cert\") pod \"ingress-canary-gbk4t\" (UID: \"c3847832-aac1-4411-8629-57ac0ba9427b\") " pod="openshift-ingress-canary/ingress-canary-gbk4t" Sep 09 02:11:15.649357 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.649318 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkjcg\" (UniqueName: \"kubernetes.io/projected/c3847832-aac1-4411-8629-57ac0ba9427b-kube-api-access-jkjcg\") pod \"ingress-canary-gbk4t\" (UID: \"c3847832-aac1-4411-8629-57ac0ba9427b\") " pod="openshift-ingress-canary/ingress-canary-gbk4t" Sep 09 02:11:15.649357 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.649357 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ec1c2f7-97f8-463a-be11-27db70b7f648-metrics-tls\") pod \"dns-default-vpmqs\" (UID: \"7ec1c2f7-97f8-463a-be11-27db70b7f648\") " pod="openshift-dns/dns-default-vpmqs" Sep 09 02:11:15.649581 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.649383 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7ec1c2f7-97f8-463a-be11-27db70b7f648-config-volume\") pod \"dns-default-vpmqs\" (UID: \"7ec1c2f7-97f8-463a-be11-27db70b7f648\") " pod="openshift-dns/dns-default-vpmqs" Sep 09 02:11:15.649581 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.649409 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ec1c2f7-97f8-463a-be11-27db70b7f648-tmp-dir\") pod \"dns-default-vpmqs\" (UID: \"7ec1c2f7-97f8-463a-be11-27db70b7f648\") " pod="openshift-dns/dns-default-vpmqs" Sep 09 02:11:15.649581 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.649432 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzdzt\" (UniqueName: \"kubernetes.io/projected/7ec1c2f7-97f8-463a-be11-27db70b7f648-kube-api-access-mzdzt\") pod \"dns-default-vpmqs\" (UID: \"7ec1c2f7-97f8-463a-be11-27db70b7f648\") " pod="openshift-dns/dns-default-vpmqs" Sep 09 02:11:15.649581 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.649458 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-data-volume\") pod \"insights-runtime-extractor-nk9tm\" (UID: \"60c1d790-25c5-4fcc-8d2b-fe69e8f309ae\") " pod="openshift-insights/insights-runtime-extractor-nk9tm" Sep 09 02:11:15.649581 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:15.649471 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 02:11:15.649581 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.649510 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-crio-socket\") pod \"insights-runtime-extractor-nk9tm\" (UID: \"60c1d790-25c5-4fcc-8d2b-fe69e8f309ae\") " pod="openshift-insights/insights-runtime-extractor-nk9tm" Sep 09 02:11:15.649581 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:15.649535 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ec1c2f7-97f8-463a-be11-27db70b7f648-metrics-tls podName:7ec1c2f7-97f8-463a-be11-27db70b7f648 nodeName:}" failed. No retries permitted until 2026-09-09 02:11:16.149516269 +0000 UTC m=+35.524547670 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7ec1c2f7-97f8-463a-be11-27db70b7f648-metrics-tls") pod "dns-default-vpmqs" (UID: "7ec1c2f7-97f8-463a-be11-27db70b7f648") : secret "dns-default-metrics-tls" not found Sep 09 02:11:15.649911 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.649625 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c3847832-aac1-4411-8629-57ac0ba9427b-cert\") pod \"ingress-canary-gbk4t\" (UID: \"c3847832-aac1-4411-8629-57ac0ba9427b\") " pod="openshift-ingress-canary/ingress-canary-gbk4t" Sep 09 02:11:15.649911 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.649649 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6b68aeda-18ee-448e-af5f-02a89181127e-signing-cabundle\") pod \"service-ca-75d99c484-c2kht\" (UID: \"6b68aeda-18ee-448e-af5f-02a89181127e\") " pod="openshift-service-ca/service-ca-75d99c484-c2kht" Sep 09 02:11:15.649911 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.649666 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nk9tm\" (UID: \"60c1d790-25c5-4fcc-8d2b-fe69e8f309ae\") " pod="openshift-insights/insights-runtime-extractor-nk9tm" Sep 09 02:11:15.649911 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.649687 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6b68aeda-18ee-448e-af5f-02a89181127e-signing-key\") pod \"service-ca-75d99c484-c2kht\" (UID: \"6b68aeda-18ee-448e-af5f-02a89181127e\") " pod="openshift-service-ca/service-ca-75d99c484-c2kht" Sep 09 02:11:15.649911 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.649711 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mq7tp\" (UniqueName: \"kubernetes.io/projected/6b68aeda-18ee-448e-af5f-02a89181127e-kube-api-access-mq7tp\") pod \"service-ca-75d99c484-c2kht\" (UID: \"6b68aeda-18ee-448e-af5f-02a89181127e\") " pod="openshift-service-ca/service-ca-75d99c484-c2kht" Sep 09 02:11:15.649911 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.649780 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ec1c2f7-97f8-463a-be11-27db70b7f648-tmp-dir\") pod \"dns-default-vpmqs\" (UID: \"7ec1c2f7-97f8-463a-be11-27db70b7f648\") " pod="openshift-dns/dns-default-vpmqs" Sep 09 02:11:15.649911 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.649782 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nk9tm\" (UID: \"60c1d790-25c5-4fcc-8d2b-fe69e8f309ae\") " pod="openshift-insights/insights-runtime-extractor-nk9tm" Sep 09 02:11:15.649911 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.649852 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jv4r\" (UniqueName: \"kubernetes.io/projected/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-kube-api-access-6jv4r\") pod \"insights-runtime-extractor-nk9tm\" (UID: \"60c1d790-25c5-4fcc-8d2b-fe69e8f309ae\") " pod="openshift-insights/insights-runtime-extractor-nk9tm" Sep 09 02:11:15.650183 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.650075 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7ec1c2f7-97f8-463a-be11-27db70b7f648-config-volume\") pod \"dns-default-vpmqs\" (UID: \"7ec1c2f7-97f8-463a-be11-27db70b7f648\") " pod="openshift-dns/dns-default-vpmqs" Sep 09 02:11:15.653702 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.653643 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c3847832-aac1-4411-8629-57ac0ba9427b-cert\") pod \"ingress-canary-gbk4t\" (UID: \"c3847832-aac1-4411-8629-57ac0ba9427b\") " pod="openshift-ingress-canary/ingress-canary-gbk4t" Sep 09 02:11:15.658057 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.658034 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkjcg\" (UniqueName: \"kubernetes.io/projected/c3847832-aac1-4411-8629-57ac0ba9427b-kube-api-access-jkjcg\") pod \"ingress-canary-gbk4t\" (UID: \"c3847832-aac1-4411-8629-57ac0ba9427b\") " pod="openshift-ingress-canary/ingress-canary-gbk4t" Sep 09 02:11:15.658194 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.658174 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzdzt\" (UniqueName: \"kubernetes.io/projected/7ec1c2f7-97f8-463a-be11-27db70b7f648-kube-api-access-mzdzt\") pod \"dns-default-vpmqs\" (UID: \"7ec1c2f7-97f8-463a-be11-27db70b7f648\") " pod="openshift-dns/dns-default-vpmqs" Sep 09 02:11:15.750387 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.750354 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nk9tm\" (UID: \"60c1d790-25c5-4fcc-8d2b-fe69e8f309ae\") " pod="openshift-insights/insights-runtime-extractor-nk9tm" Sep 09 02:11:15.750387 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.750394 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jv4r\" (UniqueName: \"kubernetes.io/projected/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-kube-api-access-6jv4r\") pod \"insights-runtime-extractor-nk9tm\" (UID: \"60c1d790-25c5-4fcc-8d2b-fe69e8f309ae\") " pod="openshift-insights/insights-runtime-extractor-nk9tm" Sep 09 02:11:15.750628 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.750457 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-data-volume\") pod \"insights-runtime-extractor-nk9tm\" (UID: \"60c1d790-25c5-4fcc-8d2b-fe69e8f309ae\") " pod="openshift-insights/insights-runtime-extractor-nk9tm" Sep 09 02:11:15.750628 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.750501 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-crio-socket\") pod \"insights-runtime-extractor-nk9tm\" (UID: \"60c1d790-25c5-4fcc-8d2b-fe69e8f309ae\") " pod="openshift-insights/insights-runtime-extractor-nk9tm" Sep 09 02:11:15.750628 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:15.750529 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 02:11:15.750628 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:15.750613 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-insights-runtime-extractor-tls podName:60c1d790-25c5-4fcc-8d2b-fe69e8f309ae nodeName:}" failed. No retries permitted until 2026-09-09 02:11:16.250590486 +0000 UTC m=+35.625621895 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nk9tm" (UID: "60c1d790-25c5-4fcc-8d2b-fe69e8f309ae") : secret "insights-runtime-extractor-tls" not found Sep 09 02:11:15.750829 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.750530 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6b68aeda-18ee-448e-af5f-02a89181127e-signing-cabundle\") pod \"service-ca-75d99c484-c2kht\" (UID: \"6b68aeda-18ee-448e-af5f-02a89181127e\") " pod="openshift-service-ca/service-ca-75d99c484-c2kht" Sep 09 02:11:15.750829 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.750712 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-crio-socket\") pod \"insights-runtime-extractor-nk9tm\" (UID: \"60c1d790-25c5-4fcc-8d2b-fe69e8f309ae\") " pod="openshift-insights/insights-runtime-extractor-nk9tm" Sep 09 02:11:15.750829 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.750723 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nk9tm\" (UID: \"60c1d790-25c5-4fcc-8d2b-fe69e8f309ae\") " pod="openshift-insights/insights-runtime-extractor-nk9tm" Sep 09 02:11:15.750829 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.750760 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6b68aeda-18ee-448e-af5f-02a89181127e-signing-key\") pod \"service-ca-75d99c484-c2kht\" (UID: \"6b68aeda-18ee-448e-af5f-02a89181127e\") " pod="openshift-service-ca/service-ca-75d99c484-c2kht" Sep 09 02:11:15.750829 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:15.750767 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 02:11:15.750829 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.750784 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mq7tp\" (UniqueName: \"kubernetes.io/projected/6b68aeda-18ee-448e-af5f-02a89181127e-kube-api-access-mq7tp\") pod \"service-ca-75d99c484-c2kht\" (UID: \"6b68aeda-18ee-448e-af5f-02a89181127e\") " pod="openshift-service-ca/service-ca-75d99c484-c2kht" Sep 09 02:11:15.751081 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:15.750852 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-kube-rbac-proxy-cm podName:60c1d790-25c5-4fcc-8d2b-fe69e8f309ae nodeName:}" failed. No retries permitted until 2026-09-09 02:11:16.250840533 +0000 UTC m=+35.625871924 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nk9tm" (UID: "60c1d790-25c5-4fcc-8d2b-fe69e8f309ae") : configmap "kube-rbac-proxy" not found Sep 09 02:11:15.751260 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.751215 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-data-volume\") pod \"insights-runtime-extractor-nk9tm\" (UID: \"60c1d790-25c5-4fcc-8d2b-fe69e8f309ae\") " pod="openshift-insights/insights-runtime-extractor-nk9tm" Sep 09 02:11:15.751596 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.751574 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6b68aeda-18ee-448e-af5f-02a89181127e-signing-cabundle\") pod \"service-ca-75d99c484-c2kht\" (UID: \"6b68aeda-18ee-448e-af5f-02a89181127e\") " pod="openshift-service-ca/service-ca-75d99c484-c2kht" Sep 09 02:11:15.754034 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.753917 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6b68aeda-18ee-448e-af5f-02a89181127e-signing-key\") pod \"service-ca-75d99c484-c2kht\" (UID: \"6b68aeda-18ee-448e-af5f-02a89181127e\") " pod="openshift-service-ca/service-ca-75d99c484-c2kht" Sep 09 02:11:15.764415 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.764189 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jv4r\" (UniqueName: \"kubernetes.io/projected/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-kube-api-access-6jv4r\") pod \"insights-runtime-extractor-nk9tm\" (UID: \"60c1d790-25c5-4fcc-8d2b-fe69e8f309ae\") " pod="openshift-insights/insights-runtime-extractor-nk9tm" Sep 09 02:11:15.764508 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.764456 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mq7tp\" (UniqueName: \"kubernetes.io/projected/6b68aeda-18ee-448e-af5f-02a89181127e-kube-api-access-mq7tp\") pod \"service-ca-75d99c484-c2kht\" (UID: \"6b68aeda-18ee-448e-af5f-02a89181127e\") " pod="openshift-service-ca/service-ca-75d99c484-c2kht" Sep 09 02:11:15.793641 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.793614 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gbk4t" Sep 09 02:11:15.806864 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:15.806840 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-c2kht" Sep 09 02:11:16.105510 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:16.105473 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8x9lh" Sep 09 02:11:16.105677 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:16.105487 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4mzjj" Sep 09 02:11:16.109830 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:16.109806 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-d95pl\"" Sep 09 02:11:16.109973 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:16.109809 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 09 02:11:16.109973 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:16.109813 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 09 02:11:16.110182 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:16.110169 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fkxcl\"" Sep 09 02:11:16.110441 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:16.110427 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 09 02:11:16.130583 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:16.130482 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-c2kht"] Sep 09 02:11:16.132303 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:16.132282 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gbk4t"] Sep 09 02:11:16.137356 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:11:16.137320 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3847832_aac1_4411_8629_57ac0ba9427b.slice/crio-6840505d6b34790665b6d0a03b2490af843bcb5d6343978c5f2c81b1fca81066 WatchSource:0}: Error finding container 6840505d6b34790665b6d0a03b2490af843bcb5d6343978c5f2c81b1fca81066: Status 404 returned error can't find the container with id 6840505d6b34790665b6d0a03b2490af843bcb5d6343978c5f2c81b1fca81066 Sep 09 02:11:16.139315 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:11:16.139291 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b68aeda_18ee_448e_af5f_02a89181127e.slice/crio-9983934e821f2a8b6ccfd8b0e623d31666405e0989ed77bdb493d9f3709e8c7b WatchSource:0}: Error finding container 9983934e821f2a8b6ccfd8b0e623d31666405e0989ed77bdb493d9f3709e8c7b: Status 404 returned error can't find the container with id 9983934e821f2a8b6ccfd8b0e623d31666405e0989ed77bdb493d9f3709e8c7b Sep 09 02:11:16.154693 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:16.154666 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ec1c2f7-97f8-463a-be11-27db70b7f648-metrics-tls\") pod \"dns-default-vpmqs\" (UID: \"7ec1c2f7-97f8-463a-be11-27db70b7f648\") " pod="openshift-dns/dns-default-vpmqs" Sep 09 02:11:16.154828 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:16.154812 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 02:11:16.154875 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:16.154867 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ec1c2f7-97f8-463a-be11-27db70b7f648-metrics-tls podName:7ec1c2f7-97f8-463a-be11-27db70b7f648 nodeName:}" failed. No retries permitted until 2026-09-09 02:11:17.154850734 +0000 UTC m=+36.529882127 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7ec1c2f7-97f8-463a-be11-27db70b7f648-metrics-tls") pod "dns-default-vpmqs" (UID: "7ec1c2f7-97f8-463a-be11-27db70b7f648") : secret "dns-default-metrics-tls" not found Sep 09 02:11:16.230907 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:16.230878 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wvtrf_55dc8777-be96-4d4f-9f9f-8180a1e158fe/dns-node-resolver/0.log" Sep 09 02:11:16.255129 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:16.255097 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nk9tm\" (UID: \"60c1d790-25c5-4fcc-8d2b-fe69e8f309ae\") " pod="openshift-insights/insights-runtime-extractor-nk9tm" Sep 09 02:11:16.255282 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:16.255269 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nk9tm\" (UID: \"60c1d790-25c5-4fcc-8d2b-fe69e8f309ae\") " pod="openshift-insights/insights-runtime-extractor-nk9tm" Sep 09 02:11:16.255360 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:16.255347 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 02:11:16.255415 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:16.255361 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 02:11:16.255415 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:16.255400 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-kube-rbac-proxy-cm podName:60c1d790-25c5-4fcc-8d2b-fe69e8f309ae nodeName:}" failed. No retries permitted until 2026-09-09 02:11:17.255387535 +0000 UTC m=+36.630418921 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nk9tm" (UID: "60c1d790-25c5-4fcc-8d2b-fe69e8f309ae") : configmap "kube-rbac-proxy" not found Sep 09 02:11:16.255415 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:16.255414 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-insights-runtime-extractor-tls podName:60c1d790-25c5-4fcc-8d2b-fe69e8f309ae nodeName:}" failed. No retries permitted until 2026-09-09 02:11:17.255406694 +0000 UTC m=+36.630438081 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nk9tm" (UID: "60c1d790-25c5-4fcc-8d2b-fe69e8f309ae") : secret "insights-runtime-extractor-tls" not found Sep 09 02:11:16.362710 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:16.362676 2561 generic.go:356] "Generic (PLEG): container finished" podID="85d4dd2e-a53d-455e-a542-9ce5ebd5b479" containerID="07ad6192a55035bdf147d85f743d375fc8c2b5edbc614c2a3dfdfb22b9ee5da8" exitCode=0 Sep 09 02:11:16.362870 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:16.362754 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5g4kk" event={"ID":"85d4dd2e-a53d-455e-a542-9ce5ebd5b479","Type":"ContainerDied","Data":"07ad6192a55035bdf147d85f743d375fc8c2b5edbc614c2a3dfdfb22b9ee5da8"} Sep 09 02:11:16.363992 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:16.363969 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gbk4t" event={"ID":"c3847832-aac1-4411-8629-57ac0ba9427b","Type":"ContainerStarted","Data":"6840505d6b34790665b6d0a03b2490af843bcb5d6343978c5f2c81b1fca81066"} Sep 09 02:11:16.365006 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:16.364984 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-c2kht" event={"ID":"6b68aeda-18ee-448e-af5f-02a89181127e","Type":"ContainerStarted","Data":"9983934e821f2a8b6ccfd8b0e623d31666405e0989ed77bdb493d9f3709e8c7b"} Sep 09 02:11:17.105288 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:17.105058 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mzt99" Sep 09 02:11:17.108290 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:17.107740 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 09 02:11:17.161863 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:17.161825 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ec1c2f7-97f8-463a-be11-27db70b7f648-metrics-tls\") pod \"dns-default-vpmqs\" (UID: \"7ec1c2f7-97f8-463a-be11-27db70b7f648\") " pod="openshift-dns/dns-default-vpmqs" Sep 09 02:11:17.162100 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:17.162029 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 02:11:17.163516 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:17.163487 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ec1c2f7-97f8-463a-be11-27db70b7f648-metrics-tls podName:7ec1c2f7-97f8-463a-be11-27db70b7f648 nodeName:}" failed. No retries permitted until 2026-09-09 02:11:19.163457815 +0000 UTC m=+38.538489214 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7ec1c2f7-97f8-463a-be11-27db70b7f648-metrics-tls") pod "dns-default-vpmqs" (UID: "7ec1c2f7-97f8-463a-be11-27db70b7f648") : secret "dns-default-metrics-tls" not found Sep 09 02:11:17.197156 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:17.197128 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-z7f6v_84536d51-1f06-4aac-ba98-7773a0e01692/node-ca/0.log" Sep 09 02:11:17.262676 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:17.262640 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nk9tm\" (UID: \"60c1d790-25c5-4fcc-8d2b-fe69e8f309ae\") " pod="openshift-insights/insights-runtime-extractor-nk9tm" Sep 09 02:11:17.262868 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:17.262703 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nk9tm\" (UID: \"60c1d790-25c5-4fcc-8d2b-fe69e8f309ae\") " pod="openshift-insights/insights-runtime-extractor-nk9tm" Sep 09 02:11:17.262868 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:17.262837 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 02:11:17.262985 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:17.262885 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-insights-runtime-extractor-tls podName:60c1d790-25c5-4fcc-8d2b-fe69e8f309ae nodeName:}" failed. No retries permitted until 2026-09-09 02:11:19.262872543 +0000 UTC m=+38.637903930 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nk9tm" (UID: "60c1d790-25c5-4fcc-8d2b-fe69e8f309ae") : secret "insights-runtime-extractor-tls" not found Sep 09 02:11:17.263138 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:17.263110 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 02:11:17.263256 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:17.263181 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-kube-rbac-proxy-cm podName:60c1d790-25c5-4fcc-8d2b-fe69e8f309ae nodeName:}" failed. No retries permitted until 2026-09-09 02:11:19.263165123 +0000 UTC m=+38.638196530 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nk9tm" (UID: "60c1d790-25c5-4fcc-8d2b-fe69e8f309ae") : configmap "kube-rbac-proxy" not found Sep 09 02:11:17.371606 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:17.371469 2561 generic.go:356] "Generic (PLEG): container finished" podID="85d4dd2e-a53d-455e-a542-9ce5ebd5b479" containerID="5245fcc010311b275bfa07deb7b2ee698799e07d00874c5669fa66ea2922367f" exitCode=0 Sep 09 02:11:17.371606 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:17.371508 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5g4kk" event={"ID":"85d4dd2e-a53d-455e-a542-9ce5ebd5b479","Type":"ContainerDied","Data":"5245fcc010311b275bfa07deb7b2ee698799e07d00874c5669fa66ea2922367f"} Sep 09 02:11:18.378443 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:18.378214 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5g4kk" event={"ID":"85d4dd2e-a53d-455e-a542-9ce5ebd5b479","Type":"ContainerStarted","Data":"adb48ba136ff780c9a41446b915c3506e7f8f3220456a8dfc3236a941b034f58"} Sep 09 02:11:18.403596 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:18.403520 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-5g4kk" podStartSLOduration=7.360576145 podStartE2EDuration="36.403503039s" totalImagesPullingTime="29.042926894s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 02:10:42 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:11:18.402234011 +0000 UTC m=+37.777265420" watchObservedRunningTime="2026-09-09 02:11:18.403503039 +0000 UTC m=+37.778534469" Sep 09 02:11:19.177789 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:19.177751 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ec1c2f7-97f8-463a-be11-27db70b7f648-metrics-tls\") pod \"dns-default-vpmqs\" (UID: \"7ec1c2f7-97f8-463a-be11-27db70b7f648\") " pod="openshift-dns/dns-default-vpmqs" Sep 09 02:11:19.177941 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:19.177899 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 02:11:19.177983 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:19.177965 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ec1c2f7-97f8-463a-be11-27db70b7f648-metrics-tls podName:7ec1c2f7-97f8-463a-be11-27db70b7f648 nodeName:}" failed. No retries permitted until 2026-09-09 02:11:23.17794947 +0000 UTC m=+42.552980857 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7ec1c2f7-97f8-463a-be11-27db70b7f648-metrics-tls") pod "dns-default-vpmqs" (UID: "7ec1c2f7-97f8-463a-be11-27db70b7f648") : secret "dns-default-metrics-tls" not found Sep 09 02:11:19.279090 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:19.279052 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nk9tm\" (UID: \"60c1d790-25c5-4fcc-8d2b-fe69e8f309ae\") " pod="openshift-insights/insights-runtime-extractor-nk9tm" Sep 09 02:11:19.279238 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:19.279122 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nk9tm\" (UID: \"60c1d790-25c5-4fcc-8d2b-fe69e8f309ae\") " pod="openshift-insights/insights-runtime-extractor-nk9tm" Sep 09 02:11:19.279238 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:19.279221 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 02:11:19.279311 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:19.279275 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-insights-runtime-extractor-tls podName:60c1d790-25c5-4fcc-8d2b-fe69e8f309ae nodeName:}" failed. No retries permitted until 2026-09-09 02:11:23.279261262 +0000 UTC m=+42.654292648 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nk9tm" (UID: "60c1d790-25c5-4fcc-8d2b-fe69e8f309ae") : secret "insights-runtime-extractor-tls" not found Sep 09 02:11:19.279593 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:19.279573 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nk9tm\" (UID: \"60c1d790-25c5-4fcc-8d2b-fe69e8f309ae\") " pod="openshift-insights/insights-runtime-extractor-nk9tm" Sep 09 02:11:20.384942 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:20.384906 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-c2kht" event={"ID":"6b68aeda-18ee-448e-af5f-02a89181127e","Type":"ContainerStarted","Data":"42010852baf21ec23a83a54b8c244a9e076f92fa732fe64adef6ac72940cb56d"} Sep 09 02:11:20.386469 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:20.386447 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gbk4t" event={"ID":"c3847832-aac1-4411-8629-57ac0ba9427b","Type":"ContainerStarted","Data":"cab31590b5d6396487c100f94bc7834daf0d3c56a5ae3f71dd5d0370f0437359"} Sep 09 02:11:20.401327 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:20.401282 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-75d99c484-c2kht" podStartSLOduration=2.224009153 podStartE2EDuration="5.401266106s" totalImagesPullingTime="3.177256953s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:11:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:11:20.400210778 +0000 UTC m=+39.775242188" watchObservedRunningTime="2026-09-09 02:11:20.401266106 +0000 UTC m=+39.776297535" Sep 09 02:11:20.415328 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:20.415291 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gbk4t" podStartSLOduration=2.244111935 podStartE2EDuration="5.415279749s" totalImagesPullingTime="3.171167814s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:11:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:11:20.414888811 +0000 UTC m=+39.789920212" watchObservedRunningTime="2026-09-09 02:11:20.415279749 +0000 UTC m=+39.790311139" Sep 09 02:11:23.207252 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:23.207216 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ec1c2f7-97f8-463a-be11-27db70b7f648-metrics-tls\") pod \"dns-default-vpmqs\" (UID: \"7ec1c2f7-97f8-463a-be11-27db70b7f648\") " pod="openshift-dns/dns-default-vpmqs" Sep 09 02:11:23.207630 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:23.207369 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 02:11:23.207630 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:23.207428 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ec1c2f7-97f8-463a-be11-27db70b7f648-metrics-tls podName:7ec1c2f7-97f8-463a-be11-27db70b7f648 nodeName:}" failed. No retries permitted until 2026-09-09 02:11:31.207411512 +0000 UTC m=+50.582442903 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7ec1c2f7-97f8-463a-be11-27db70b7f648-metrics-tls") pod "dns-default-vpmqs" (UID: "7ec1c2f7-97f8-463a-be11-27db70b7f648") : secret "dns-default-metrics-tls" not found Sep 09 02:11:23.307666 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:23.307632 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nk9tm\" (UID: \"60c1d790-25c5-4fcc-8d2b-fe69e8f309ae\") " pod="openshift-insights/insights-runtime-extractor-nk9tm" Sep 09 02:11:23.307836 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:23.307741 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 02:11:23.307836 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:23.307789 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-insights-runtime-extractor-tls podName:60c1d790-25c5-4fcc-8d2b-fe69e8f309ae nodeName:}" failed. No retries permitted until 2026-09-09 02:11:31.307777121 +0000 UTC m=+50.682808508 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nk9tm" (UID: "60c1d790-25c5-4fcc-8d2b-fe69e8f309ae") : secret "insights-runtime-extractor-tls" not found Sep 09 02:11:31.268755 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:31.268710 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ec1c2f7-97f8-463a-be11-27db70b7f648-metrics-tls\") pod \"dns-default-vpmqs\" (UID: \"7ec1c2f7-97f8-463a-be11-27db70b7f648\") " pod="openshift-dns/dns-default-vpmqs" Sep 09 02:11:31.272243 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:31.272209 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ec1c2f7-97f8-463a-be11-27db70b7f648-metrics-tls\") pod \"dns-default-vpmqs\" (UID: \"7ec1c2f7-97f8-463a-be11-27db70b7f648\") " pod="openshift-dns/dns-default-vpmqs" Sep 09 02:11:31.369320 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:31.369286 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nk9tm\" (UID: \"60c1d790-25c5-4fcc-8d2b-fe69e8f309ae\") " pod="openshift-insights/insights-runtime-extractor-nk9tm" Sep 09 02:11:31.371461 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:31.371437 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/60c1d790-25c5-4fcc-8d2b-fe69e8f309ae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nk9tm\" (UID: \"60c1d790-25c5-4fcc-8d2b-fe69e8f309ae\") " pod="openshift-insights/insights-runtime-extractor-nk9tm" Sep 09 02:11:31.371576 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:31.371540 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vpmqs" Sep 09 02:11:31.431234 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:31.431206 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nk9tm" Sep 09 02:11:31.675341 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:31.675311 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vpmqs"] Sep 09 02:11:31.679329 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:11:31.679306 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ec1c2f7_97f8_463a_be11_27db70b7f648.slice/crio-628fb7593b347864395b9d6569929cb3ead426376678722d50e1174318c6d4cd WatchSource:0}: Error finding container 628fb7593b347864395b9d6569929cb3ead426376678722d50e1174318c6d4cd: Status 404 returned error can't find the container with id 628fb7593b347864395b9d6569929cb3ead426376678722d50e1174318c6d4cd Sep 09 02:11:31.720309 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:31.720284 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nk9tm"] Sep 09 02:11:31.724184 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:11:31.724151 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60c1d790_25c5_4fcc_8d2b_fe69e8f309ae.slice/crio-5f2cc83c0482169af6fa1cfc708e48fcbd22819abe23747daad4e9994b266c4c WatchSource:0}: Error finding container 5f2cc83c0482169af6fa1cfc708e48fcbd22819abe23747daad4e9994b266c4c: Status 404 returned error can't find the container with id 5f2cc83c0482169af6fa1cfc708e48fcbd22819abe23747daad4e9994b266c4c Sep 09 02:11:32.419511 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:32.419470 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vpmqs" event={"ID":"7ec1c2f7-97f8-463a-be11-27db70b7f648","Type":"ContainerStarted","Data":"628fb7593b347864395b9d6569929cb3ead426376678722d50e1174318c6d4cd"} Sep 09 02:11:32.422075 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:32.422042 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nk9tm" event={"ID":"60c1d790-25c5-4fcc-8d2b-fe69e8f309ae","Type":"ContainerStarted","Data":"403cc708046210c1fbdf2b5c41612a6cd329ad5a30abc470a2591403e4450bde"} Sep 09 02:11:32.422216 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:32.422100 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nk9tm" event={"ID":"60c1d790-25c5-4fcc-8d2b-fe69e8f309ae","Type":"ContainerStarted","Data":"5f2cc83c0482169af6fa1cfc708e48fcbd22819abe23747daad4e9994b266c4c"} Sep 09 02:11:33.426717 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:33.426685 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vpmqs" event={"ID":"7ec1c2f7-97f8-463a-be11-27db70b7f648","Type":"ContainerStarted","Data":"9938e3e8cfdfd3f13fadaeeadeb567346149c46d8cccec9bbff4e8fc089015fe"} Sep 09 02:11:33.428957 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:33.428920 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nk9tm" event={"ID":"60c1d790-25c5-4fcc-8d2b-fe69e8f309ae","Type":"ContainerStarted","Data":"773ec4229aa6ad6006eea292898405be4562f689d138f19c3184c9a4776e067c"} Sep 09 02:11:34.436993 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:34.436959 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nk9tm" event={"ID":"60c1d790-25c5-4fcc-8d2b-fe69e8f309ae","Type":"ContainerStarted","Data":"b936f7727b2c06d337dc3d12b8657dc0ca85018a852d462ea6b8c0b67893093c"} Sep 09 02:11:34.438784 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:34.438762 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vpmqs" event={"ID":"7ec1c2f7-97f8-463a-be11-27db70b7f648","Type":"ContainerStarted","Data":"d2823e3fe61cebd9c485bb6f38a3114cf5146556c156ca8eef4922c787c8ed27"} Sep 09 02:11:34.438909 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:34.438893 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vpmqs" Sep 09 02:11:34.455400 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:34.455362 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-nk9tm" podStartSLOduration=17.166095659 podStartE2EDuration="19.455349252s" totalImagesPullingTime="2.289253593s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 02:11:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:11:34.455196939 +0000 UTC m=+53.830228350" watchObservedRunningTime="2026-09-09 02:11:34.455349252 +0000 UTC m=+53.830380661" Sep 09 02:11:34.471557 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:34.471521 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vpmqs" podStartSLOduration=17.827332407 podStartE2EDuration="19.471509096s" totalImagesPullingTime="1.644176689s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:11:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:11:34.47132332 +0000 UTC m=+53.846354730" watchObservedRunningTime="2026-09-09 02:11:34.471509096 +0000 UTC m=+53.846540504" Sep 09 02:11:35.536831 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.536800 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dfcdd8f64-59l8w"] Sep 09 02:11:35.538624 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.538606 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-666cccd549-vgl7f"] Sep 09 02:11:35.538768 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.538749 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dfcdd8f64-59l8w" Sep 09 02:11:35.540683 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.540663 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-666cccd549-vgl7f" Sep 09 02:11:35.540800 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.540778 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-pcr5c\"" Sep 09 02:11:35.541276 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.541258 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 09 02:11:35.541540 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.541520 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Sep 09 02:11:35.541616 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.541557 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 09 02:11:35.541616 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.541603 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 09 02:11:35.542967 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.542950 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Sep 09 02:11:35.543306 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.543291 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Sep 09 02:11:35.544959 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.544940 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Sep 09 02:11:35.545306 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.545286 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Sep 09 02:11:35.554554 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.554500 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dfcdd8f64-59l8w"] Sep 09 02:11:35.555264 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.555245 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-666cccd549-vgl7f"] Sep 09 02:11:35.703868 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.703828 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/2633bce1-5470-4922-b969-2aa7e6904a9f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-666cccd549-vgl7f\" (UID: \"2633bce1-5470-4922-b969-2aa7e6904a9f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-666cccd549-vgl7f" Sep 09 02:11:35.703868 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.703872 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xqpz\" (UniqueName: \"kubernetes.io/projected/2633bce1-5470-4922-b969-2aa7e6904a9f-kube-api-access-4xqpz\") pod \"cluster-proxy-proxy-agent-666cccd549-vgl7f\" (UID: \"2633bce1-5470-4922-b969-2aa7e6904a9f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-666cccd549-vgl7f" Sep 09 02:11:35.704110 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.703951 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7d9c352c-e103-4b79-8b5d-7c6620f446e6-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5dfcdd8f64-59l8w\" (UID: \"7d9c352c-e103-4b79-8b5d-7c6620f446e6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dfcdd8f64-59l8w" Sep 09 02:11:35.704110 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.703998 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/2633bce1-5470-4922-b969-2aa7e6904a9f-ca\") pod \"cluster-proxy-proxy-agent-666cccd549-vgl7f\" (UID: \"2633bce1-5470-4922-b969-2aa7e6904a9f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-666cccd549-vgl7f" Sep 09 02:11:35.704110 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.704040 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/2633bce1-5470-4922-b969-2aa7e6904a9f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-666cccd549-vgl7f\" (UID: \"2633bce1-5470-4922-b969-2aa7e6904a9f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-666cccd549-vgl7f" Sep 09 02:11:35.704110 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.704072 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xpcl\" (UniqueName: \"kubernetes.io/projected/7d9c352c-e103-4b79-8b5d-7c6620f446e6-kube-api-access-4xpcl\") pod \"managed-serviceaccount-addon-agent-5dfcdd8f64-59l8w\" (UID: \"7d9c352c-e103-4b79-8b5d-7c6620f446e6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dfcdd8f64-59l8w" Sep 09 02:11:35.704110 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.704099 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2633bce1-5470-4922-b969-2aa7e6904a9f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-666cccd549-vgl7f\" (UID: \"2633bce1-5470-4922-b969-2aa7e6904a9f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-666cccd549-vgl7f" Sep 09 02:11:35.704268 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.704161 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/2633bce1-5470-4922-b969-2aa7e6904a9f-hub\") pod \"cluster-proxy-proxy-agent-666cccd549-vgl7f\" (UID: \"2633bce1-5470-4922-b969-2aa7e6904a9f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-666cccd549-vgl7f" Sep 09 02:11:35.805152 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.805069 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/2633bce1-5470-4922-b969-2aa7e6904a9f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-666cccd549-vgl7f\" (UID: \"2633bce1-5470-4922-b969-2aa7e6904a9f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-666cccd549-vgl7f" Sep 09 02:11:35.805152 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.805102 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xqpz\" (UniqueName: \"kubernetes.io/projected/2633bce1-5470-4922-b969-2aa7e6904a9f-kube-api-access-4xqpz\") pod \"cluster-proxy-proxy-agent-666cccd549-vgl7f\" (UID: \"2633bce1-5470-4922-b969-2aa7e6904a9f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-666cccd549-vgl7f" Sep 09 02:11:35.805152 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.805129 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7d9c352c-e103-4b79-8b5d-7c6620f446e6-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5dfcdd8f64-59l8w\" (UID: \"7d9c352c-e103-4b79-8b5d-7c6620f446e6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dfcdd8f64-59l8w" Sep 09 02:11:35.805152 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.805146 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/2633bce1-5470-4922-b969-2aa7e6904a9f-ca\") pod \"cluster-proxy-proxy-agent-666cccd549-vgl7f\" (UID: \"2633bce1-5470-4922-b969-2aa7e6904a9f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-666cccd549-vgl7f" Sep 09 02:11:35.805447 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.805165 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/2633bce1-5470-4922-b969-2aa7e6904a9f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-666cccd549-vgl7f\" (UID: \"2633bce1-5470-4922-b969-2aa7e6904a9f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-666cccd549-vgl7f" Sep 09 02:11:35.805447 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.805196 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xpcl\" (UniqueName: \"kubernetes.io/projected/7d9c352c-e103-4b79-8b5d-7c6620f446e6-kube-api-access-4xpcl\") pod \"managed-serviceaccount-addon-agent-5dfcdd8f64-59l8w\" (UID: \"7d9c352c-e103-4b79-8b5d-7c6620f446e6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dfcdd8f64-59l8w" Sep 09 02:11:35.805447 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.805224 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2633bce1-5470-4922-b969-2aa7e6904a9f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-666cccd549-vgl7f\" (UID: \"2633bce1-5470-4922-b969-2aa7e6904a9f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-666cccd549-vgl7f" Sep 09 02:11:35.805447 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.805269 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/2633bce1-5470-4922-b969-2aa7e6904a9f-hub\") pod \"cluster-proxy-proxy-agent-666cccd549-vgl7f\" (UID: \"2633bce1-5470-4922-b969-2aa7e6904a9f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-666cccd549-vgl7f" Sep 09 02:11:35.806074 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.806051 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/2633bce1-5470-4922-b969-2aa7e6904a9f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-666cccd549-vgl7f\" (UID: \"2633bce1-5470-4922-b969-2aa7e6904a9f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-666cccd549-vgl7f" Sep 09 02:11:35.807798 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.807778 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/2633bce1-5470-4922-b969-2aa7e6904a9f-hub\") pod \"cluster-proxy-proxy-agent-666cccd549-vgl7f\" (UID: \"2633bce1-5470-4922-b969-2aa7e6904a9f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-666cccd549-vgl7f" Sep 09 02:11:35.808165 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.808133 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/2633bce1-5470-4922-b969-2aa7e6904a9f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-666cccd549-vgl7f\" (UID: \"2633bce1-5470-4922-b969-2aa7e6904a9f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-666cccd549-vgl7f" Sep 09 02:11:35.808254 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.808135 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/2633bce1-5470-4922-b969-2aa7e6904a9f-ca\") pod \"cluster-proxy-proxy-agent-666cccd549-vgl7f\" (UID: \"2633bce1-5470-4922-b969-2aa7e6904a9f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-666cccd549-vgl7f" Sep 09 02:11:35.808344 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.808317 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2633bce1-5470-4922-b969-2aa7e6904a9f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-666cccd549-vgl7f\" (UID: \"2633bce1-5470-4922-b969-2aa7e6904a9f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-666cccd549-vgl7f" Sep 09 02:11:35.808378 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.808334 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7d9c352c-e103-4b79-8b5d-7c6620f446e6-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5dfcdd8f64-59l8w\" (UID: \"7d9c352c-e103-4b79-8b5d-7c6620f446e6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dfcdd8f64-59l8w" Sep 09 02:11:35.815568 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.815544 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xqpz\" (UniqueName: \"kubernetes.io/projected/2633bce1-5470-4922-b969-2aa7e6904a9f-kube-api-access-4xqpz\") pod \"cluster-proxy-proxy-agent-666cccd549-vgl7f\" (UID: \"2633bce1-5470-4922-b969-2aa7e6904a9f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-666cccd549-vgl7f" Sep 09 02:11:35.815848 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.815828 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xpcl\" (UniqueName: \"kubernetes.io/projected/7d9c352c-e103-4b79-8b5d-7c6620f446e6-kube-api-access-4xpcl\") pod \"managed-serviceaccount-addon-agent-5dfcdd8f64-59l8w\" (UID: \"7d9c352c-e103-4b79-8b5d-7c6620f446e6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dfcdd8f64-59l8w" Sep 09 02:11:35.856766 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.856732 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dfcdd8f64-59l8w" Sep 09 02:11:35.862459 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:35.862439 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-666cccd549-vgl7f" Sep 09 02:11:36.129428 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:36.129360 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dfcdd8f64-59l8w"] Sep 09 02:11:36.132712 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:11:36.132667 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7d9c352c_e103_4b79_8b5d_7c6620f446e6.slice/crio-83370d20743d74add2f4d07168101b3bfeebe94419ba550bca82b978306cd905 WatchSource:0}: Error finding container 83370d20743d74add2f4d07168101b3bfeebe94419ba550bca82b978306cd905: Status 404 returned error can't find the container with id 83370d20743d74add2f4d07168101b3bfeebe94419ba550bca82b978306cd905 Sep 09 02:11:36.133393 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:36.133373 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-666cccd549-vgl7f"] Sep 09 02:11:36.136213 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:11:36.136195 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2633bce1_5470_4922_b969_2aa7e6904a9f.slice/crio-ed05cf25d24fa7c0080e2f030b35856d64bdc179f21b338e8f7fa843f5a3995c WatchSource:0}: Error finding container ed05cf25d24fa7c0080e2f030b35856d64bdc179f21b338e8f7fa843f5a3995c: Status 404 returned error can't find the container with id ed05cf25d24fa7c0080e2f030b35856d64bdc179f21b338e8f7fa843f5a3995c Sep 09 02:11:36.449742 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:36.449705 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dfcdd8f64-59l8w" event={"ID":"7d9c352c-e103-4b79-8b5d-7c6620f446e6","Type":"ContainerStarted","Data":"83370d20743d74add2f4d07168101b3bfeebe94419ba550bca82b978306cd905"} Sep 09 02:11:36.450798 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:36.450780 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-666cccd549-vgl7f" event={"ID":"2633bce1-5470-4922-b969-2aa7e6904a9f","Type":"ContainerStarted","Data":"ed05cf25d24fa7c0080e2f030b35856d64bdc179f21b338e8f7fa843f5a3995c"} Sep 09 02:11:40.466070 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:40.466030 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dfcdd8f64-59l8w" event={"ID":"7d9c352c-e103-4b79-8b5d-7c6620f446e6","Type":"ContainerStarted","Data":"ebb9ceed2b24ec26b86f0d07b8d175129f52b42ae4e5858110058c3fa9678013"} Sep 09 02:11:40.482642 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:40.482586 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5dfcdd8f64-59l8w" podStartSLOduration=1.468074556 podStartE2EDuration="5.48256576s" totalImagesPullingTime="4.014491204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:11:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:11:40.481231742 +0000 UTC m=+59.856263368" watchObservedRunningTime="2026-09-09 02:11:40.48256576 +0000 UTC m=+59.857597169" Sep 09 02:11:41.359132 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:41.359098 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9cxcz" Sep 09 02:11:44.444710 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:44.444677 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vpmqs" Sep 09 02:11:45.485418 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:45.485383 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-666cccd549-vgl7f" event={"ID":"2633bce1-5470-4922-b969-2aa7e6904a9f","Type":"ContainerStarted","Data":"beaaf8ab6e4f9bb5cbfe18d363725d6554e6fe2902374fd4296869476f4853a5"} Sep 09 02:11:45.485418 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:45.485425 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-666cccd549-vgl7f" event={"ID":"2633bce1-5470-4922-b969-2aa7e6904a9f","Type":"ContainerStarted","Data":"9c0053cd58c41fb78d3935e3d4b20559ac5083985a22b91440c5a43f5334f249"} Sep 09 02:11:45.485948 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:45.485437 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-666cccd549-vgl7f" event={"ID":"2633bce1-5470-4922-b969-2aa7e6904a9f","Type":"ContainerStarted","Data":"6f1f6dd45acdec63e857a9d7f65f3af76db1e92d1cc48cc340b9052163524cfd"} Sep 09 02:11:45.509255 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:45.509207 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-666cccd549-vgl7f" podStartSLOduration=1.993450398 podStartE2EDuration="10.509191161s" totalImagesPullingTime="8.515740763s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:11:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:11:45.507457299 +0000 UTC m=+64.882488708" watchObservedRunningTime="2026-09-09 02:11:45.509191161 +0000 UTC m=+64.884222569" Sep 09 02:11:46.892456 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:46.892417 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129-original-pull-secret\") pod \"global-pull-secret-syncer-mzt99\" (UID: \"0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129\") " pod="kube-system/global-pull-secret-syncer-mzt99" Sep 09 02:11:46.892456 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:46.892460 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a06a9649-a313-401a-80cb-c71fddc9d349-metrics-certs\") pod \"network-metrics-daemon-8x9lh\" (UID: \"a06a9649-a313-401a-80cb-c71fddc9d349\") " pod="openshift-multus/network-metrics-daemon-8x9lh" Sep 09 02:11:46.892913 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:46.892482 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4456s\" (UniqueName: \"kubernetes.io/projected/ae5099dc-0b2e-4a01-887c-3076a88ecbd8-kube-api-access-4456s\") pod \"network-check-target-4mzjj\" (UID: \"ae5099dc-0b2e-4a01-887c-3076a88ecbd8\") " pod="openshift-network-diagnostics/network-check-target-4mzjj" Sep 09 02:11:46.895256 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:46.895236 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 09 02:11:46.895915 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:46.895888 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 09 02:11:46.895915 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:46.895905 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 09 02:11:46.905208 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:46.905168 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129-original-pull-secret\") pod \"global-pull-secret-syncer-mzt99\" (UID: \"0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129\") " pod="kube-system/global-pull-secret-syncer-mzt99" Sep 09 02:11:46.905294 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:46.905190 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a06a9649-a313-401a-80cb-c71fddc9d349-metrics-certs\") pod \"network-metrics-daemon-8x9lh\" (UID: \"a06a9649-a313-401a-80cb-c71fddc9d349\") " pod="openshift-multus/network-metrics-daemon-8x9lh" Sep 09 02:11:46.905555 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:46.905540 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 09 02:11:46.915534 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:46.915516 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4456s\" (UniqueName: \"kubernetes.io/projected/ae5099dc-0b2e-4a01-887c-3076a88ecbd8-kube-api-access-4456s\") pod \"network-check-target-4mzjj\" (UID: \"ae5099dc-0b2e-4a01-887c-3076a88ecbd8\") " pod="openshift-network-diagnostics/network-check-target-4mzjj" Sep 09 02:11:47.018908 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:47.018875 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fkxcl\"" Sep 09 02:11:47.025923 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:47.025902 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-d95pl\"" Sep 09 02:11:47.026935 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:47.026920 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8x9lh" Sep 09 02:11:47.033110 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:47.033093 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4mzjj" Sep 09 02:11:47.116357 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:47.116332 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mzt99" Sep 09 02:11:47.332180 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:47.332148 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8x9lh"] Sep 09 02:11:47.336618 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:11:47.336594 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda06a9649_a313_401a_80cb_c71fddc9d349.slice/crio-46eba5c26792e24fada22e38673a2d547d77204a92d4ba82fc0e2e2de74c7a61 WatchSource:0}: Error finding container 46eba5c26792e24fada22e38673a2d547d77204a92d4ba82fc0e2e2de74c7a61: Status 404 returned error can't find the container with id 46eba5c26792e24fada22e38673a2d547d77204a92d4ba82fc0e2e2de74c7a61 Sep 09 02:11:47.366182 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:47.366121 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4mzjj"] Sep 09 02:11:47.368409 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:11:47.368387 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae5099dc_0b2e_4a01_887c_3076a88ecbd8.slice/crio-de5ce69e71214d1b988fbb8746089e00053aba6db4eee8d42bcd245ac5516b2c WatchSource:0}: Error finding container de5ce69e71214d1b988fbb8746089e00053aba6db4eee8d42bcd245ac5516b2c: Status 404 returned error can't find the container with id de5ce69e71214d1b988fbb8746089e00053aba6db4eee8d42bcd245ac5516b2c Sep 09 02:11:47.417144 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:47.417074 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mzt99"] Sep 09 02:11:47.421165 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:11:47.421143 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bf6e8eb_d36b_40d5_a3e3_9d9e148e0129.slice/crio-a40d8a65b80f7674ee288c6ed2c9b3f51a178795f1830c128a0986aad02215b8 WatchSource:0}: Error finding container a40d8a65b80f7674ee288c6ed2c9b3f51a178795f1830c128a0986aad02215b8: Status 404 returned error can't find the container with id a40d8a65b80f7674ee288c6ed2c9b3f51a178795f1830c128a0986aad02215b8 Sep 09 02:11:47.492981 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:47.492944 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4mzjj" event={"ID":"ae5099dc-0b2e-4a01-887c-3076a88ecbd8","Type":"ContainerStarted","Data":"de5ce69e71214d1b988fbb8746089e00053aba6db4eee8d42bcd245ac5516b2c"} Sep 09 02:11:47.494072 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:47.494037 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8x9lh" event={"ID":"a06a9649-a313-401a-80cb-c71fddc9d349","Type":"ContainerStarted","Data":"46eba5c26792e24fada22e38673a2d547d77204a92d4ba82fc0e2e2de74c7a61"} Sep 09 02:11:47.495150 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:47.495120 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mzt99" event={"ID":"0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129","Type":"ContainerStarted","Data":"a40d8a65b80f7674ee288c6ed2c9b3f51a178795f1830c128a0986aad02215b8"} Sep 09 02:11:48.501487 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:48.501441 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8x9lh" event={"ID":"a06a9649-a313-401a-80cb-c71fddc9d349","Type":"ContainerStarted","Data":"be8da3733144c6af421f208935c55e205d2b7fc757e9da1c537f87369fb8245e"} Sep 09 02:11:49.507633 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:49.507592 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8x9lh" event={"ID":"a06a9649-a313-401a-80cb-c71fddc9d349","Type":"ContainerStarted","Data":"994c31b2f69bbfb58517f7ca9cf2eb206bdec0497cc85ce15c23f55cfa6c2cc8"} Sep 09 02:11:49.523598 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:49.523544 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-8x9lh" podStartSLOduration=66.513362746 podStartE2EDuration="1m7.523518518s" totalImagesPullingTime="1.010155772s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:10:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:11:49.522050742 +0000 UTC m=+68.897082152" watchObservedRunningTime="2026-09-09 02:11:49.523518518 +0000 UTC m=+68.898549930" Sep 09 02:11:52.276838 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.276738 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rfctf"] Sep 09 02:11:52.279399 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.279375 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rfctf" Sep 09 02:11:52.282536 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.282491 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 09 02:11:52.282536 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.282506 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 09 02:11:52.282843 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.282826 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 09 02:11:52.283244 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.283198 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 09 02:11:52.283664 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.283508 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 09 02:11:52.283664 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.283558 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 09 02:11:52.283802 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.283708 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-q7c6n\"" Sep 09 02:11:52.285499 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.285459 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-4fr2r"] Sep 09 02:11:52.288200 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.288180 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-4fr2r" Sep 09 02:11:52.290494 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.290357 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Sep 09 02:11:52.290494 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.290380 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Sep 09 02:11:52.290968 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.290949 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-hhdm7\"" Sep 09 02:11:52.291734 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.291714 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Sep 09 02:11:52.305360 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.305336 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-4fr2r"] Sep 09 02:11:52.331278 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.331247 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/235e6e09-3eb3-4733-896d-c5245a0415f2-root\") pod \"node-exporter-rfctf\" (UID: \"235e6e09-3eb3-4733-896d-c5245a0415f2\") " pod="openshift-monitoring/node-exporter-rfctf" Sep 09 02:11:52.331419 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.331284 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/235e6e09-3eb3-4733-896d-c5245a0415f2-node-exporter-wtmp\") pod \"node-exporter-rfctf\" (UID: \"235e6e09-3eb3-4733-896d-c5245a0415f2\") " pod="openshift-monitoring/node-exporter-rfctf" Sep 09 02:11:52.331419 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.331310 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrc5w\" (UniqueName: \"kubernetes.io/projected/a66f722e-21be-4b13-9eb4-acef95b3b4ae-kube-api-access-xrc5w\") pod \"kube-state-metrics-74cb57df7-4fr2r\" (UID: \"a66f722e-21be-4b13-9eb4-acef95b3b4ae\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-4fr2r" Sep 09 02:11:52.331419 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.331333 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a66f722e-21be-4b13-9eb4-acef95b3b4ae-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-4fr2r\" (UID: \"a66f722e-21be-4b13-9eb4-acef95b3b4ae\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-4fr2r" Sep 09 02:11:52.331419 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.331381 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/235e6e09-3eb3-4733-896d-c5245a0415f2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rfctf\" (UID: \"235e6e09-3eb3-4733-896d-c5245a0415f2\") " pod="openshift-monitoring/node-exporter-rfctf" Sep 09 02:11:52.331419 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.331407 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/235e6e09-3eb3-4733-896d-c5245a0415f2-node-exporter-tls\") pod \"node-exporter-rfctf\" (UID: \"235e6e09-3eb3-4733-896d-c5245a0415f2\") " pod="openshift-monitoring/node-exporter-rfctf" Sep 09 02:11:52.331632 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.331449 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/235e6e09-3eb3-4733-896d-c5245a0415f2-node-exporter-accelerators-collector-config\") pod \"node-exporter-rfctf\" (UID: \"235e6e09-3eb3-4733-896d-c5245a0415f2\") " pod="openshift-monitoring/node-exporter-rfctf" Sep 09 02:11:52.331632 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.331476 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptzsh\" (UniqueName: \"kubernetes.io/projected/235e6e09-3eb3-4733-896d-c5245a0415f2-kube-api-access-ptzsh\") pod \"node-exporter-rfctf\" (UID: \"235e6e09-3eb3-4733-896d-c5245a0415f2\") " pod="openshift-monitoring/node-exporter-rfctf" Sep 09 02:11:52.331632 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.331498 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a66f722e-21be-4b13-9eb4-acef95b3b4ae-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-4fr2r\" (UID: \"a66f722e-21be-4b13-9eb4-acef95b3b4ae\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-4fr2r" Sep 09 02:11:52.331632 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.331582 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a66f722e-21be-4b13-9eb4-acef95b3b4ae-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-4fr2r\" (UID: \"a66f722e-21be-4b13-9eb4-acef95b3b4ae\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-4fr2r" Sep 09 02:11:52.331749 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.331634 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a66f722e-21be-4b13-9eb4-acef95b3b4ae-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-4fr2r\" (UID: \"a66f722e-21be-4b13-9eb4-acef95b3b4ae\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-4fr2r" Sep 09 02:11:52.331749 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.331668 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/235e6e09-3eb3-4733-896d-c5245a0415f2-metrics-client-ca\") pod \"node-exporter-rfctf\" (UID: \"235e6e09-3eb3-4733-896d-c5245a0415f2\") " pod="openshift-monitoring/node-exporter-rfctf" Sep 09 02:11:52.331812 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.331715 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/235e6e09-3eb3-4733-896d-c5245a0415f2-node-exporter-textfile\") pod \"node-exporter-rfctf\" (UID: \"235e6e09-3eb3-4733-896d-c5245a0415f2\") " pod="openshift-monitoring/node-exporter-rfctf" Sep 09 02:11:52.331812 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.331783 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a66f722e-21be-4b13-9eb4-acef95b3b4ae-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-4fr2r\" (UID: \"a66f722e-21be-4b13-9eb4-acef95b3b4ae\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-4fr2r" Sep 09 02:11:52.331874 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.331822 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/235e6e09-3eb3-4733-896d-c5245a0415f2-sys\") pod \"node-exporter-rfctf\" (UID: \"235e6e09-3eb3-4733-896d-c5245a0415f2\") " pod="openshift-monitoring/node-exporter-rfctf" Sep 09 02:11:52.432328 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.432283 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/235e6e09-3eb3-4733-896d-c5245a0415f2-node-exporter-wtmp\") pod \"node-exporter-rfctf\" (UID: \"235e6e09-3eb3-4733-896d-c5245a0415f2\") " pod="openshift-monitoring/node-exporter-rfctf" Sep 09 02:11:52.432328 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.432330 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrc5w\" (UniqueName: \"kubernetes.io/projected/a66f722e-21be-4b13-9eb4-acef95b3b4ae-kube-api-access-xrc5w\") pod \"kube-state-metrics-74cb57df7-4fr2r\" (UID: \"a66f722e-21be-4b13-9eb4-acef95b3b4ae\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-4fr2r" Sep 09 02:11:52.432571 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.432463 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a66f722e-21be-4b13-9eb4-acef95b3b4ae-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-4fr2r\" (UID: \"a66f722e-21be-4b13-9eb4-acef95b3b4ae\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-4fr2r" Sep 09 02:11:52.432571 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.432533 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/235e6e09-3eb3-4733-896d-c5245a0415f2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rfctf\" (UID: \"235e6e09-3eb3-4733-896d-c5245a0415f2\") " pod="openshift-monitoring/node-exporter-rfctf" Sep 09 02:11:52.432671 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.432580 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/235e6e09-3eb3-4733-896d-c5245a0415f2-node-exporter-tls\") pod \"node-exporter-rfctf\" (UID: \"235e6e09-3eb3-4733-896d-c5245a0415f2\") " pod="openshift-monitoring/node-exporter-rfctf" Sep 09 02:11:52.432671 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.432653 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/235e6e09-3eb3-4733-896d-c5245a0415f2-node-exporter-accelerators-collector-config\") pod \"node-exporter-rfctf\" (UID: \"235e6e09-3eb3-4733-896d-c5245a0415f2\") " pod="openshift-monitoring/node-exporter-rfctf" Sep 09 02:11:52.432767 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.432687 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptzsh\" (UniqueName: \"kubernetes.io/projected/235e6e09-3eb3-4733-896d-c5245a0415f2-kube-api-access-ptzsh\") pod \"node-exporter-rfctf\" (UID: \"235e6e09-3eb3-4733-896d-c5245a0415f2\") " pod="openshift-monitoring/node-exporter-rfctf" Sep 09 02:11:52.432767 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.432710 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a66f722e-21be-4b13-9eb4-acef95b3b4ae-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-4fr2r\" (UID: \"a66f722e-21be-4b13-9eb4-acef95b3b4ae\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-4fr2r" Sep 09 02:11:52.432767 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.432751 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a66f722e-21be-4b13-9eb4-acef95b3b4ae-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-4fr2r\" (UID: \"a66f722e-21be-4b13-9eb4-acef95b3b4ae\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-4fr2r" Sep 09 02:11:52.432904 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.432785 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a66f722e-21be-4b13-9eb4-acef95b3b4ae-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-4fr2r\" (UID: \"a66f722e-21be-4b13-9eb4-acef95b3b4ae\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-4fr2r" Sep 09 02:11:52.432904 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.432831 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/235e6e09-3eb3-4733-896d-c5245a0415f2-metrics-client-ca\") pod \"node-exporter-rfctf\" (UID: \"235e6e09-3eb3-4733-896d-c5245a0415f2\") " pod="openshift-monitoring/node-exporter-rfctf" Sep 09 02:11:52.432904 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.432862 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/235e6e09-3eb3-4733-896d-c5245a0415f2-node-exporter-textfile\") pod \"node-exporter-rfctf\" (UID: \"235e6e09-3eb3-4733-896d-c5245a0415f2\") " pod="openshift-monitoring/node-exporter-rfctf" Sep 09 02:11:52.432904 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.432898 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a66f722e-21be-4b13-9eb4-acef95b3b4ae-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-4fr2r\" (UID: \"a66f722e-21be-4b13-9eb4-acef95b3b4ae\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-4fr2r" Sep 09 02:11:52.433115 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.432915 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a66f722e-21be-4b13-9eb4-acef95b3b4ae-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-4fr2r\" (UID: \"a66f722e-21be-4b13-9eb4-acef95b3b4ae\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-4fr2r" Sep 09 02:11:52.433115 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.432922 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/235e6e09-3eb3-4733-896d-c5245a0415f2-sys\") pod \"node-exporter-rfctf\" (UID: \"235e6e09-3eb3-4733-896d-c5245a0415f2\") " pod="openshift-monitoring/node-exporter-rfctf" Sep 09 02:11:52.433115 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.432982 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/235e6e09-3eb3-4733-896d-c5245a0415f2-root\") pod \"node-exporter-rfctf\" (UID: \"235e6e09-3eb3-4733-896d-c5245a0415f2\") " pod="openshift-monitoring/node-exporter-rfctf" Sep 09 02:11:52.433293 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.432486 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/235e6e09-3eb3-4733-896d-c5245a0415f2-node-exporter-wtmp\") pod \"node-exporter-rfctf\" (UID: \"235e6e09-3eb3-4733-896d-c5245a0415f2\") " pod="openshift-monitoring/node-exporter-rfctf" Sep 09 02:11:52.433787 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:52.433526 2561 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Sep 09 02:11:52.433787 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:11:52.433591 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a66f722e-21be-4b13-9eb4-acef95b3b4ae-kube-state-metrics-tls podName:a66f722e-21be-4b13-9eb4-acef95b3b4ae nodeName:}" failed. No retries permitted until 2026-09-09 02:11:52.933573608 +0000 UTC m=+72.308605015 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/a66f722e-21be-4b13-9eb4-acef95b3b4ae-kube-state-metrics-tls") pod "kube-state-metrics-74cb57df7-4fr2r" (UID: "a66f722e-21be-4b13-9eb4-acef95b3b4ae") : secret "kube-state-metrics-tls" not found Sep 09 02:11:52.433787 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.433678 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/235e6e09-3eb3-4733-896d-c5245a0415f2-node-exporter-textfile\") pod \"node-exporter-rfctf\" (UID: \"235e6e09-3eb3-4733-896d-c5245a0415f2\") " pod="openshift-monitoring/node-exporter-rfctf" Sep 09 02:11:52.433787 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.433695 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/235e6e09-3eb3-4733-896d-c5245a0415f2-node-exporter-accelerators-collector-config\") pod \"node-exporter-rfctf\" (UID: \"235e6e09-3eb3-4733-896d-c5245a0415f2\") " pod="openshift-monitoring/node-exporter-rfctf" Sep 09 02:11:52.433787 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.433714 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a66f722e-21be-4b13-9eb4-acef95b3b4ae-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-4fr2r\" (UID: \"a66f722e-21be-4b13-9eb4-acef95b3b4ae\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-4fr2r" Sep 09 02:11:52.433787 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.433772 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/235e6e09-3eb3-4733-896d-c5245a0415f2-sys\") pod \"node-exporter-rfctf\" (UID: \"235e6e09-3eb3-4733-896d-c5245a0415f2\") " pod="openshift-monitoring/node-exporter-rfctf" Sep 09 02:11:52.434199 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.433998 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a66f722e-21be-4b13-9eb4-acef95b3b4ae-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-4fr2r\" (UID: \"a66f722e-21be-4b13-9eb4-acef95b3b4ae\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-4fr2r" Sep 09 02:11:52.434199 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.434098 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/235e6e09-3eb3-4733-896d-c5245a0415f2-root\") pod \"node-exporter-rfctf\" (UID: \"235e6e09-3eb3-4733-896d-c5245a0415f2\") " pod="openshift-monitoring/node-exporter-rfctf" Sep 09 02:11:52.434441 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.434417 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/235e6e09-3eb3-4733-896d-c5245a0415f2-metrics-client-ca\") pod \"node-exporter-rfctf\" (UID: \"235e6e09-3eb3-4733-896d-c5245a0415f2\") " pod="openshift-monitoring/node-exporter-rfctf" Sep 09 02:11:52.435881 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.435824 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/235e6e09-3eb3-4733-896d-c5245a0415f2-node-exporter-tls\") pod \"node-exporter-rfctf\" (UID: \"235e6e09-3eb3-4733-896d-c5245a0415f2\") " pod="openshift-monitoring/node-exporter-rfctf" Sep 09 02:11:52.435881 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.435838 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/235e6e09-3eb3-4733-896d-c5245a0415f2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rfctf\" (UID: \"235e6e09-3eb3-4733-896d-c5245a0415f2\") " pod="openshift-monitoring/node-exporter-rfctf" Sep 09 02:11:52.436075 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.436055 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a66f722e-21be-4b13-9eb4-acef95b3b4ae-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-4fr2r\" (UID: \"a66f722e-21be-4b13-9eb4-acef95b3b4ae\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-4fr2r" Sep 09 02:11:52.441534 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.441513 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptzsh\" (UniqueName: \"kubernetes.io/projected/235e6e09-3eb3-4733-896d-c5245a0415f2-kube-api-access-ptzsh\") pod \"node-exporter-rfctf\" (UID: \"235e6e09-3eb3-4733-896d-c5245a0415f2\") " pod="openshift-monitoring/node-exporter-rfctf" Sep 09 02:11:52.442407 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.442386 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrc5w\" (UniqueName: \"kubernetes.io/projected/a66f722e-21be-4b13-9eb4-acef95b3b4ae-kube-api-access-xrc5w\") pod \"kube-state-metrics-74cb57df7-4fr2r\" (UID: \"a66f722e-21be-4b13-9eb4-acef95b3b4ae\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-4fr2r" Sep 09 02:11:52.520479 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.520446 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mzt99" event={"ID":"0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129","Type":"ContainerStarted","Data":"1bf2f312a2e3593ec53f0dc5cee8b3bcb2ce7d51d4cdcc8d4a7b0921bdc3ab3b"} Sep 09 02:11:52.522847 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.522816 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4mzjj" event={"ID":"ae5099dc-0b2e-4a01-887c-3076a88ecbd8","Type":"ContainerStarted","Data":"70001c3f2a3cacd122e3342097fecc22d00b4fa38e2c1b07ca046702ebb1f66f"} Sep 09 02:11:52.522989 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.522953 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-4mzjj" Sep 09 02:11:52.536486 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.536389 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-mzt99" podStartSLOduration=65.969371595 podStartE2EDuration="1m10.536374116s" totalImagesPullingTime="4.567002521s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 02:10:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:11:52.535597975 +0000 UTC m=+71.910629396" watchObservedRunningTime="2026-09-09 02:11:52.536374116 +0000 UTC m=+71.911405525" Sep 09 02:11:52.552566 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.552519 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-4mzjj" podStartSLOduration=66.932923025 podStartE2EDuration="1m11.552503902s" totalImagesPullingTime="4.619580877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:10:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:11:52.551243217 +0000 UTC m=+71.926274625" watchObservedRunningTime="2026-09-09 02:11:52.552503902 +0000 UTC m=+71.927535311" Sep 09 02:11:52.592684 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.592656 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rfctf" Sep 09 02:11:52.610747 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:11:52.610717 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod235e6e09_3eb3_4733_896d_c5245a0415f2.slice/crio-a8956dde74fa0b0de98e27c3e350d7ab900d64a0470d5a0db9f42af0b8af380e WatchSource:0}: Error finding container a8956dde74fa0b0de98e27c3e350d7ab900d64a0470d5a0db9f42af0b8af380e: Status 404 returned error can't find the container with id a8956dde74fa0b0de98e27c3e350d7ab900d64a0470d5a0db9f42af0b8af380e Sep 09 02:11:52.936387 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.936350 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a66f722e-21be-4b13-9eb4-acef95b3b4ae-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-4fr2r\" (UID: \"a66f722e-21be-4b13-9eb4-acef95b3b4ae\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-4fr2r" Sep 09 02:11:52.938758 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:52.938726 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a66f722e-21be-4b13-9eb4-acef95b3b4ae-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-4fr2r\" (UID: \"a66f722e-21be-4b13-9eb4-acef95b3b4ae\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-4fr2r" Sep 09 02:11:53.206234 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:53.206154 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-4fr2r" Sep 09 02:11:53.499782 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:53.499700 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-4fr2r"] Sep 09 02:11:53.528348 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:53.528313 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rfctf" event={"ID":"235e6e09-3eb3-4733-896d-c5245a0415f2","Type":"ContainerStarted","Data":"a8956dde74fa0b0de98e27c3e350d7ab900d64a0470d5a0db9f42af0b8af380e"} Sep 09 02:11:53.799074 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:11:53.798979 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda66f722e_21be_4b13_9eb4_acef95b3b4ae.slice/crio-1fc95dfe3fd9257a29c348577946399e4b4d67eba834dee26fc2555de582782f WatchSource:0}: Error finding container 1fc95dfe3fd9257a29c348577946399e4b4d67eba834dee26fc2555de582782f: Status 404 returned error can't find the container with id 1fc95dfe3fd9257a29c348577946399e4b4d67eba834dee26fc2555de582782f Sep 09 02:11:54.533988 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:54.533955 2561 generic.go:356] "Generic (PLEG): container finished" podID="235e6e09-3eb3-4733-896d-c5245a0415f2" containerID="d6dcde2124ccbbfa280c037c195614a71c1436f225122044f4c56633348039e4" exitCode=0 Sep 09 02:11:54.534422 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:54.534045 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rfctf" event={"ID":"235e6e09-3eb3-4733-896d-c5245a0415f2","Type":"ContainerDied","Data":"d6dcde2124ccbbfa280c037c195614a71c1436f225122044f4c56633348039e4"} Sep 09 02:11:54.535322 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:54.535304 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-4fr2r" event={"ID":"a66f722e-21be-4b13-9eb4-acef95b3b4ae","Type":"ContainerStarted","Data":"1fc95dfe3fd9257a29c348577946399e4b4d67eba834dee26fc2555de582782f"} Sep 09 02:11:55.540668 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:55.540638 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rfctf" event={"ID":"235e6e09-3eb3-4733-896d-c5245a0415f2","Type":"ContainerStarted","Data":"3a9a14c7551783865c168c2807669472697dd794349783b147ac30c8251b2510"} Sep 09 02:11:55.541185 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:55.540700 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rfctf" event={"ID":"235e6e09-3eb3-4733-896d-c5245a0415f2","Type":"ContainerStarted","Data":"9c47441b1275fb0342497f27fd19b4e9e3fb2b62ea5385033fb8d8eba3ee9579"} Sep 09 02:11:55.565715 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:55.565669 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rfctf" podStartSLOduration=2.218150719 podStartE2EDuration="3.565655503s" totalImagesPullingTime="1.286743751s" totalInitContainerRuntime="60.761033ms" isStatefulPod=true podCreationTimestamp="2026-09-09 02:11:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:11:55.564050399 +0000 UTC m=+74.939081808" watchObservedRunningTime="2026-09-09 02:11:55.565655503 +0000 UTC m=+74.940686936" Sep 09 02:11:56.551426 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:56.551389 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-4fr2r" event={"ID":"a66f722e-21be-4b13-9eb4-acef95b3b4ae","Type":"ContainerStarted","Data":"6715ac65cce04e821d85e6ff3b09797776f4b1e10f4b42f9cb4ab078afca5071"} Sep 09 02:11:56.551795 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:56.551456 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-4fr2r" event={"ID":"a66f722e-21be-4b13-9eb4-acef95b3b4ae","Type":"ContainerStarted","Data":"e5ddcee1cc32a76e5cf96fffbbe9fdb8d652d5be74336715f2a6b062ba1d062f"} Sep 09 02:11:56.551795 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:56.551476 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-4fr2r" event={"ID":"a66f722e-21be-4b13-9eb4-acef95b3b4ae","Type":"ContainerStarted","Data":"ccec9013258801243c6f546045ea1984a0beef986b7a563e3bf8f55cd2f8fe5f"} Sep 09 02:11:56.578038 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:56.577544 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-4fr2r" podStartSLOduration=2.885797272 podStartE2EDuration="4.577522362s" totalImagesPullingTime="1.69172509s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:11:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:11:56.576650862 +0000 UTC m=+75.951682271" watchObservedRunningTime="2026-09-09 02:11:56.577522362 +0000 UTC m=+75.952553771" Sep 09 02:11:58.485869 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.485831 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 02:11:58.488513 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.488495 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.490823 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.490800 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 09 02:11:58.490991 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.490974 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 09 02:11:58.493439 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.493415 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 09 02:11:58.493648 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.493630 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 09 02:11:58.493710 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.493681 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 09 02:11:58.493783 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.493764 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 09 02:11:58.493829 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.493787 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 09 02:11:58.493887 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.493873 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 09 02:11:58.493947 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.493926 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 09 02:11:58.494000 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.493980 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 09 02:11:58.494393 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.494374 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-s7x67\"" Sep 09 02:11:58.494491 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.494406 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 09 02:11:58.494491 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.494458 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 09 02:11:58.494596 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.494517 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-9q75ht698hars\"" Sep 09 02:11:58.494655 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.494608 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 09 02:11:58.501089 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.501067 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 09 02:11:58.501994 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.501977 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 09 02:11:58.507471 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.507450 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 02:11:58.580549 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.580518 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.580549 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.580551 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/651a2e91-648c-465e-8726-94efaff2bbf2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.580745 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.580576 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hd5v4\" (UniqueName: \"kubernetes.io/projected/651a2e91-648c-465e-8726-94efaff2bbf2-kube-api-access-hd5v4\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.580745 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.580630 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.580745 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.580669 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.580745 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.580725 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.580906 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.580763 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.580906 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.580786 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.580906 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.580800 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.580906 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.580818 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-config\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.580906 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.580861 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.580906 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.580894 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/651a2e91-648c-465e-8726-94efaff2bbf2-config-out\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.581113 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.580944 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.581113 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.580960 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.581113 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.580975 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.581113 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.581033 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.581113 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.581062 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.581113 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.581082 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.581113 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.581098 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-web-config\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.581316 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.581121 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/651a2e91-648c-465e-8726-94efaff2bbf2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.681632 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.681596 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.681632 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.681631 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.681856 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.681656 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-config\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.681856 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.681680 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.681856 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.681707 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/651a2e91-648c-465e-8726-94efaff2bbf2-config-out\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.681856 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.681744 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.681856 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.681764 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.681856 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.681786 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.681856 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.681820 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.681856 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.681845 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.682262 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.681879 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.682262 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.681903 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-web-config\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.682262 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.681928 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/651a2e91-648c-465e-8726-94efaff2bbf2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.682262 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.681961 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.682262 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.681989 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/651a2e91-648c-465e-8726-94efaff2bbf2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.682262 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.682055 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hd5v4\" (UniqueName: \"kubernetes.io/projected/651a2e91-648c-465e-8726-94efaff2bbf2-kube-api-access-hd5v4\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.682262 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.682082 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.682262 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.682122 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.682262 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.682170 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.682262 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.682203 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.682747 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.682372 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.683760 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.683428 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/651a2e91-648c-465e-8726-94efaff2bbf2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.683760 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.683637 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.684747 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.684173 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.685501 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.685477 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.686476 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.686447 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/651a2e91-648c-465e-8726-94efaff2bbf2-config-out\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.686566 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.686502 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.687174 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.687146 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.687455 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.687431 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.688271 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.688240 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.688731 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.688702 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.688816 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.688758 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.689524 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.689072 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.689524 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.689258 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/651a2e91-648c-465e-8726-94efaff2bbf2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.689524 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.689485 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-config\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.689734 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.689607 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.690034 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.689987 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.690392 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.690368 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-web-config\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.691174 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.691155 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.693030 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.692990 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hd5v4\" (UniqueName: \"kubernetes.io/projected/651a2e91-648c-465e-8726-94efaff2bbf2-kube-api-access-hd5v4\") pod \"prometheus-k8s-0\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:58.802332 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:58.802240 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:11:59.087953 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:59.087928 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 02:11:59.090356 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:11:59.090324 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod651a2e91_648c_465e_8726_94efaff2bbf2.slice/crio-bb99d054410029ab919512aa0602f4438cd1480b75a7041538fb28df2b8e2a70 WatchSource:0}: Error finding container bb99d054410029ab919512aa0602f4438cd1480b75a7041538fb28df2b8e2a70: Status 404 returned error can't find the container with id bb99d054410029ab919512aa0602f4438cd1480b75a7041538fb28df2b8e2a70 Sep 09 02:11:59.563032 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:11:59.562979 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"651a2e91-648c-465e-8726-94efaff2bbf2","Type":"ContainerStarted","Data":"bb99d054410029ab919512aa0602f4438cd1480b75a7041538fb28df2b8e2a70"} Sep 09 02:12:00.567693 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:12:00.567656 2561 generic.go:356] "Generic (PLEG): container finished" podID="651a2e91-648c-465e-8726-94efaff2bbf2" containerID="99cfedc36e31101740f02272f980a801b0a48c269b0974b5ef7ea20f440bbd06" exitCode=0 Sep 09 02:12:00.568061 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:12:00.567745 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"651a2e91-648c-465e-8726-94efaff2bbf2","Type":"ContainerDied","Data":"99cfedc36e31101740f02272f980a801b0a48c269b0974b5ef7ea20f440bbd06"} Sep 09 02:12:04.588234 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:12:04.588192 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"651a2e91-648c-465e-8726-94efaff2bbf2","Type":"ContainerStarted","Data":"56887768ecd4e1fbbbfa3428aa45d0adfae3bd39670faa1522cf0b0dbe9847df"} Sep 09 02:12:04.588715 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:12:04.588287 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"651a2e91-648c-465e-8726-94efaff2bbf2","Type":"ContainerStarted","Data":"a731d9ba654e8291d9ec39a49510de31d2236cbadb1e259af6123a6e0c517bb3"} Sep 09 02:12:06.601813 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:12:06.601772 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"651a2e91-648c-465e-8726-94efaff2bbf2","Type":"ContainerStarted","Data":"00be1a39065c82998b7ee3588fe21f26f342436e82a220f2cc344165521970cc"} Sep 09 02:12:06.602197 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:12:06.601841 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"651a2e91-648c-465e-8726-94efaff2bbf2","Type":"ContainerStarted","Data":"cb19d1ce2ae6e7eff7a5b969f5fdcd3578fa5a67c7a9a13f03265926f57fd9c2"} Sep 09 02:12:06.602197 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:12:06.601884 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"651a2e91-648c-465e-8726-94efaff2bbf2","Type":"ContainerStarted","Data":"2226fdee248bca22e7b63106c223dba65148c0bbd8b347f92c3322d5bb32a638"} Sep 09 02:12:06.602197 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:12:06.601908 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"651a2e91-648c-465e-8726-94efaff2bbf2","Type":"ContainerStarted","Data":"d2ac0c17949726b2ae1168c05b6b72a306c53679330bf3471e7dc9a9657310d4"} Sep 09 02:12:06.632436 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:12:06.632367 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.6922905889999997 podStartE2EDuration="8.632345175s" totalImagesPullingTime="5.921633162s" totalInitContainerRuntime="18.421424ms" isStatefulPod=false podCreationTimestamp="2026-09-09 02:11:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:12:06.630525602 +0000 UTC m=+86.005557010" watchObservedRunningTime="2026-09-09 02:12:06.632345175 +0000 UTC m=+86.007376586" Sep 09 02:12:08.803364 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:12:08.803320 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:12:23.531472 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:12:23.531443 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-4mzjj" Sep 09 02:12:58.802587 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:12:58.802552 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:12:58.821508 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:12:58.821482 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:12:59.808876 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:12:59.808847 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:16.921126 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:16.921091 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 02:13:16.921551 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:16.921386 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="651a2e91-648c-465e-8726-94efaff2bbf2" containerName="prometheus" containerID="cri-o://a731d9ba654e8291d9ec39a49510de31d2236cbadb1e259af6123a6e0c517bb3" gracePeriod=600 Sep 09 02:13:16.921551 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:16.921443 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="651a2e91-648c-465e-8726-94efaff2bbf2" containerName="kube-rbac-proxy-web" containerID="cri-o://2226fdee248bca22e7b63106c223dba65148c0bbd8b347f92c3322d5bb32a638" gracePeriod=600 Sep 09 02:13:16.921551 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:16.921468 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="651a2e91-648c-465e-8726-94efaff2bbf2" containerName="config-reloader" containerID="cri-o://56887768ecd4e1fbbbfa3428aa45d0adfae3bd39670faa1522cf0b0dbe9847df" gracePeriod=600 Sep 09 02:13:16.921551 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:16.921439 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="651a2e91-648c-465e-8726-94efaff2bbf2" containerName="thanos-sidecar" containerID="cri-o://d2ac0c17949726b2ae1168c05b6b72a306c53679330bf3471e7dc9a9657310d4" gracePeriod=600 Sep 09 02:13:16.921551 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:16.921424 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="651a2e91-648c-465e-8726-94efaff2bbf2" containerName="kube-rbac-proxy" containerID="cri-o://cb19d1ce2ae6e7eff7a5b969f5fdcd3578fa5a67c7a9a13f03265926f57fd9c2" gracePeriod=600 Sep 09 02:13:16.921841 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:16.921470 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="651a2e91-648c-465e-8726-94efaff2bbf2" containerName="kube-rbac-proxy-thanos" containerID="cri-o://00be1a39065c82998b7ee3588fe21f26f342436e82a220f2cc344165521970cc" gracePeriod=600 Sep 09 02:13:17.159666 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.159642 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:17.186471 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.186398 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-secret-metrics-client-certs\") pod \"651a2e91-648c-465e-8726-94efaff2bbf2\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " Sep 09 02:13:17.186471 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.186434 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-secret-kube-rbac-proxy\") pod \"651a2e91-648c-465e-8726-94efaff2bbf2\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " Sep 09 02:13:17.186471 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.186458 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/651a2e91-648c-465e-8726-94efaff2bbf2-tls-assets\") pod \"651a2e91-648c-465e-8726-94efaff2bbf2\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " Sep 09 02:13:17.186471 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.186475 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-prometheus-trusted-ca-bundle\") pod \"651a2e91-648c-465e-8726-94efaff2bbf2\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " Sep 09 02:13:17.186725 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.186494 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hd5v4\" (UniqueName: \"kubernetes.io/projected/651a2e91-648c-465e-8726-94efaff2bbf2-kube-api-access-hd5v4\") pod \"651a2e91-648c-465e-8726-94efaff2bbf2\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " Sep 09 02:13:17.186725 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.186513 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-prometheus-k8s-rulefiles-1\") pod \"651a2e91-648c-465e-8726-94efaff2bbf2\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " Sep 09 02:13:17.186725 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.186535 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-secret-prometheus-k8s-tls\") pod \"651a2e91-648c-465e-8726-94efaff2bbf2\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " Sep 09 02:13:17.186725 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.186561 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-secret-grpc-tls\") pod \"651a2e91-648c-465e-8726-94efaff2bbf2\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " Sep 09 02:13:17.186725 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.186591 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/651a2e91-648c-465e-8726-94efaff2bbf2-prometheus-k8s-db\") pod \"651a2e91-648c-465e-8726-94efaff2bbf2\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " Sep 09 02:13:17.186725 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.186657 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"651a2e91-648c-465e-8726-94efaff2bbf2\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " Sep 09 02:13:17.186725 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.186687 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-thanos-prometheus-http-client-file\") pod \"651a2e91-648c-465e-8726-94efaff2bbf2\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " Sep 09 02:13:17.186725 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.186712 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-web-config\") pod \"651a2e91-648c-465e-8726-94efaff2bbf2\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " Sep 09 02:13:17.187134 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.186737 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-configmap-metrics-client-ca\") pod \"651a2e91-648c-465e-8726-94efaff2bbf2\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " Sep 09 02:13:17.187134 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.186771 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-configmap-serving-certs-ca-bundle\") pod \"651a2e91-648c-465e-8726-94efaff2bbf2\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " Sep 09 02:13:17.187134 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.186805 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/651a2e91-648c-465e-8726-94efaff2bbf2-config-out\") pod \"651a2e91-648c-465e-8726-94efaff2bbf2\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " Sep 09 02:13:17.187134 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.186831 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"651a2e91-648c-465e-8726-94efaff2bbf2\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " Sep 09 02:13:17.187134 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.186862 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-prometheus-k8s-rulefiles-0\") pod \"651a2e91-648c-465e-8726-94efaff2bbf2\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " Sep 09 02:13:17.187134 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.186881 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "651a2e91-648c-465e-8726-94efaff2bbf2" (UID: "651a2e91-648c-465e-8726-94efaff2bbf2"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 02:13:17.187134 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.186914 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-prometheus-k8s-rulefiles-2\") pod \"651a2e91-648c-465e-8726-94efaff2bbf2\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " Sep 09 02:13:17.187134 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.186943 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "651a2e91-648c-465e-8726-94efaff2bbf2" (UID: "651a2e91-648c-465e-8726-94efaff2bbf2"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 02:13:17.187134 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.186958 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-configmap-kubelet-serving-ca-bundle\") pod \"651a2e91-648c-465e-8726-94efaff2bbf2\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " Sep 09 02:13:17.187134 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.187036 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-config\") pod \"651a2e91-648c-465e-8726-94efaff2bbf2\" (UID: \"651a2e91-648c-465e-8726-94efaff2bbf2\") " Sep 09 02:13:17.187634 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.187322 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "651a2e91-648c-465e-8726-94efaff2bbf2" (UID: "651a2e91-648c-465e-8726-94efaff2bbf2"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 02:13:17.187634 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.187609 2561 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:13:17.187634 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.187632 2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-prometheus-trusted-ca-bundle\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:13:17.187789 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.187649 2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:13:17.188996 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.188874 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/651a2e91-648c-465e-8726-94efaff2bbf2-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "651a2e91-648c-465e-8726-94efaff2bbf2" (UID: "651a2e91-648c-465e-8726-94efaff2bbf2"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 02:13:17.189539 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.189510 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "651a2e91-648c-465e-8726-94efaff2bbf2" (UID: "651a2e91-648c-465e-8726-94efaff2bbf2"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 02:13:17.189881 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.189849 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "651a2e91-648c-465e-8726-94efaff2bbf2" (UID: "651a2e91-648c-465e-8726-94efaff2bbf2"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 02:13:17.189881 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.189847 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "651a2e91-648c-465e-8726-94efaff2bbf2" (UID: "651a2e91-648c-465e-8726-94efaff2bbf2"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 02:13:17.190050 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.189922 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/651a2e91-648c-465e-8726-94efaff2bbf2-kube-api-access-hd5v4" (OuterVolumeSpecName: "kube-api-access-hd5v4") pod "651a2e91-648c-465e-8726-94efaff2bbf2" (UID: "651a2e91-648c-465e-8726-94efaff2bbf2"). InnerVolumeSpecName "kube-api-access-hd5v4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 02:13:17.190193 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.190170 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "651a2e91-648c-465e-8726-94efaff2bbf2" (UID: "651a2e91-648c-465e-8726-94efaff2bbf2"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 02:13:17.190777 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.190740 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "651a2e91-648c-465e-8726-94efaff2bbf2" (UID: "651a2e91-648c-465e-8726-94efaff2bbf2"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 02:13:17.191209 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.191176 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/651a2e91-648c-465e-8726-94efaff2bbf2-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "651a2e91-648c-465e-8726-94efaff2bbf2" (UID: "651a2e91-648c-465e-8726-94efaff2bbf2"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 02:13:17.191300 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.191265 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "651a2e91-648c-465e-8726-94efaff2bbf2" (UID: "651a2e91-648c-465e-8726-94efaff2bbf2"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 02:13:17.191614 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.191588 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "651a2e91-648c-465e-8726-94efaff2bbf2" (UID: "651a2e91-648c-465e-8726-94efaff2bbf2"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 02:13:17.192149 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.192093 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "651a2e91-648c-465e-8726-94efaff2bbf2" (UID: "651a2e91-648c-465e-8726-94efaff2bbf2"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 02:13:17.192238 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.192210 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-config" (OuterVolumeSpecName: "config") pod "651a2e91-648c-465e-8726-94efaff2bbf2" (UID: "651a2e91-648c-465e-8726-94efaff2bbf2"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 02:13:17.192357 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.192333 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "651a2e91-648c-465e-8726-94efaff2bbf2" (UID: "651a2e91-648c-465e-8726-94efaff2bbf2"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 02:13:17.192414 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.192334 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/651a2e91-648c-465e-8726-94efaff2bbf2-config-out" (OuterVolumeSpecName: "config-out") pod "651a2e91-648c-465e-8726-94efaff2bbf2" (UID: "651a2e91-648c-465e-8726-94efaff2bbf2"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 02:13:17.193035 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.192996 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "651a2e91-648c-465e-8726-94efaff2bbf2" (UID: "651a2e91-648c-465e-8726-94efaff2bbf2"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 02:13:17.193335 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.193308 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "651a2e91-648c-465e-8726-94efaff2bbf2" (UID: "651a2e91-648c-465e-8726-94efaff2bbf2"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 02:13:17.201811 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.201778 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-web-config" (OuterVolumeSpecName: "web-config") pod "651a2e91-648c-465e-8726-94efaff2bbf2" (UID: "651a2e91-648c-465e-8726-94efaff2bbf2"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 02:13:17.288313 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.288265 2561 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/651a2e91-648c-465e-8726-94efaff2bbf2-config-out\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:13:17.288313 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.288310 2561 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:13:17.288313 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.288322 2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:13:17.288560 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.288333 2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:13:17.288560 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.288343 2561 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-config\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:13:17.288560 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.288351 2561 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-secret-metrics-client-certs\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:13:17.288560 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.288360 2561 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-secret-kube-rbac-proxy\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:13:17.288560 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.288369 2561 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/651a2e91-648c-465e-8726-94efaff2bbf2-tls-assets\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:13:17.288560 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.288377 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hd5v4\" (UniqueName: \"kubernetes.io/projected/651a2e91-648c-465e-8726-94efaff2bbf2-kube-api-access-hd5v4\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:13:17.288560 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.288386 2561 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-secret-prometheus-k8s-tls\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:13:17.288560 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.288393 2561 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-secret-grpc-tls\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:13:17.288560 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.288401 2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/651a2e91-648c-465e-8726-94efaff2bbf2-prometheus-k8s-db\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:13:17.288560 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.288410 2561 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:13:17.288560 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.288420 2561 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-thanos-prometheus-http-client-file\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:13:17.288560 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.288428 2561 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/651a2e91-648c-465e-8726-94efaff2bbf2-web-config\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:13:17.288560 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.288436 2561 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-configmap-metrics-client-ca\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:13:17.288560 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.288444 2561 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/651a2e91-648c-465e-8726-94efaff2bbf2-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:13:17.863382 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.863348 2561 generic.go:356] "Generic (PLEG): container finished" podID="651a2e91-648c-465e-8726-94efaff2bbf2" containerID="00be1a39065c82998b7ee3588fe21f26f342436e82a220f2cc344165521970cc" exitCode=0 Sep 09 02:13:17.863382 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.863372 2561 generic.go:356] "Generic (PLEG): container finished" podID="651a2e91-648c-465e-8726-94efaff2bbf2" containerID="cb19d1ce2ae6e7eff7a5b969f5fdcd3578fa5a67c7a9a13f03265926f57fd9c2" exitCode=0 Sep 09 02:13:17.863382 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.863379 2561 generic.go:356] "Generic (PLEG): container finished" podID="651a2e91-648c-465e-8726-94efaff2bbf2" containerID="2226fdee248bca22e7b63106c223dba65148c0bbd8b347f92c3322d5bb32a638" exitCode=0 Sep 09 02:13:17.863382 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.863385 2561 generic.go:356] "Generic (PLEG): container finished" podID="651a2e91-648c-465e-8726-94efaff2bbf2" containerID="d2ac0c17949726b2ae1168c05b6b72a306c53679330bf3471e7dc9a9657310d4" exitCode=0 Sep 09 02:13:17.863382 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.863390 2561 generic.go:356] "Generic (PLEG): container finished" podID="651a2e91-648c-465e-8726-94efaff2bbf2" containerID="56887768ecd4e1fbbbfa3428aa45d0adfae3bd39670faa1522cf0b0dbe9847df" exitCode=0 Sep 09 02:13:17.863382 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.863395 2561 generic.go:356] "Generic (PLEG): container finished" podID="651a2e91-648c-465e-8726-94efaff2bbf2" containerID="a731d9ba654e8291d9ec39a49510de31d2236cbadb1e259af6123a6e0c517bb3" exitCode=0 Sep 09 02:13:17.863866 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.863431 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"651a2e91-648c-465e-8726-94efaff2bbf2","Type":"ContainerDied","Data":"00be1a39065c82998b7ee3588fe21f26f342436e82a220f2cc344165521970cc"} Sep 09 02:13:17.863866 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.863448 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:17.863866 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.863527 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"651a2e91-648c-465e-8726-94efaff2bbf2","Type":"ContainerDied","Data":"cb19d1ce2ae6e7eff7a5b969f5fdcd3578fa5a67c7a9a13f03265926f57fd9c2"} Sep 09 02:13:17.863866 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.863556 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"651a2e91-648c-465e-8726-94efaff2bbf2","Type":"ContainerDied","Data":"2226fdee248bca22e7b63106c223dba65148c0bbd8b347f92c3322d5bb32a638"} Sep 09 02:13:17.863866 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.863580 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"651a2e91-648c-465e-8726-94efaff2bbf2","Type":"ContainerDied","Data":"d2ac0c17949726b2ae1168c05b6b72a306c53679330bf3471e7dc9a9657310d4"} Sep 09 02:13:17.863866 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.863594 2561 scope.go:117] "RemoveContainer" containerID="00be1a39065c82998b7ee3588fe21f26f342436e82a220f2cc344165521970cc" Sep 09 02:13:17.863866 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.863605 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"651a2e91-648c-465e-8726-94efaff2bbf2","Type":"ContainerDied","Data":"56887768ecd4e1fbbbfa3428aa45d0adfae3bd39670faa1522cf0b0dbe9847df"} Sep 09 02:13:17.863866 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.863631 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"651a2e91-648c-465e-8726-94efaff2bbf2","Type":"ContainerDied","Data":"a731d9ba654e8291d9ec39a49510de31d2236cbadb1e259af6123a6e0c517bb3"} Sep 09 02:13:17.863866 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.863655 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"651a2e91-648c-465e-8726-94efaff2bbf2","Type":"ContainerDied","Data":"bb99d054410029ab919512aa0602f4438cd1480b75a7041538fb28df2b8e2a70"} Sep 09 02:13:17.871420 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.871326 2561 scope.go:117] "RemoveContainer" containerID="cb19d1ce2ae6e7eff7a5b969f5fdcd3578fa5a67c7a9a13f03265926f57fd9c2" Sep 09 02:13:17.878557 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.878540 2561 scope.go:117] "RemoveContainer" containerID="2226fdee248bca22e7b63106c223dba65148c0bbd8b347f92c3322d5bb32a638" Sep 09 02:13:17.885159 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.885139 2561 scope.go:117] "RemoveContainer" containerID="d2ac0c17949726b2ae1168c05b6b72a306c53679330bf3471e7dc9a9657310d4" Sep 09 02:13:17.889130 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.889106 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 02:13:17.893031 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.892997 2561 scope.go:117] "RemoveContainer" containerID="56887768ecd4e1fbbbfa3428aa45d0adfae3bd39670faa1522cf0b0dbe9847df" Sep 09 02:13:17.893918 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.893896 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 02:13:17.899976 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.899959 2561 scope.go:117] "RemoveContainer" containerID="a731d9ba654e8291d9ec39a49510de31d2236cbadb1e259af6123a6e0c517bb3" Sep 09 02:13:17.907060 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.907009 2561 scope.go:117] "RemoveContainer" containerID="99cfedc36e31101740f02272f980a801b0a48c269b0974b5ef7ea20f440bbd06" Sep 09 02:13:17.914195 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.914173 2561 scope.go:117] "RemoveContainer" containerID="00be1a39065c82998b7ee3588fe21f26f342436e82a220f2cc344165521970cc" Sep 09 02:13:17.914493 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:13:17.914467 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00be1a39065c82998b7ee3588fe21f26f342436e82a220f2cc344165521970cc\": container with ID starting with 00be1a39065c82998b7ee3588fe21f26f342436e82a220f2cc344165521970cc not found: ID does not exist" containerID="00be1a39065c82998b7ee3588fe21f26f342436e82a220f2cc344165521970cc" Sep 09 02:13:17.914580 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.914501 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00be1a39065c82998b7ee3588fe21f26f342436e82a220f2cc344165521970cc"} err="failed to get container status \"00be1a39065c82998b7ee3588fe21f26f342436e82a220f2cc344165521970cc\": rpc error: code = NotFound desc = could not find container \"00be1a39065c82998b7ee3588fe21f26f342436e82a220f2cc344165521970cc\": container with ID starting with 00be1a39065c82998b7ee3588fe21f26f342436e82a220f2cc344165521970cc not found: ID does not exist" Sep 09 02:13:17.914580 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.914538 2561 scope.go:117] "RemoveContainer" containerID="cb19d1ce2ae6e7eff7a5b969f5fdcd3578fa5a67c7a9a13f03265926f57fd9c2" Sep 09 02:13:17.914795 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:13:17.914780 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb19d1ce2ae6e7eff7a5b969f5fdcd3578fa5a67c7a9a13f03265926f57fd9c2\": container with ID starting with cb19d1ce2ae6e7eff7a5b969f5fdcd3578fa5a67c7a9a13f03265926f57fd9c2 not found: ID does not exist" containerID="cb19d1ce2ae6e7eff7a5b969f5fdcd3578fa5a67c7a9a13f03265926f57fd9c2" Sep 09 02:13:17.914836 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.914798 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb19d1ce2ae6e7eff7a5b969f5fdcd3578fa5a67c7a9a13f03265926f57fd9c2"} err="failed to get container status \"cb19d1ce2ae6e7eff7a5b969f5fdcd3578fa5a67c7a9a13f03265926f57fd9c2\": rpc error: code = NotFound desc = could not find container \"cb19d1ce2ae6e7eff7a5b969f5fdcd3578fa5a67c7a9a13f03265926f57fd9c2\": container with ID starting with cb19d1ce2ae6e7eff7a5b969f5fdcd3578fa5a67c7a9a13f03265926f57fd9c2 not found: ID does not exist" Sep 09 02:13:17.914836 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.914811 2561 scope.go:117] "RemoveContainer" containerID="2226fdee248bca22e7b63106c223dba65148c0bbd8b347f92c3322d5bb32a638" Sep 09 02:13:17.915003 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:13:17.914988 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2226fdee248bca22e7b63106c223dba65148c0bbd8b347f92c3322d5bb32a638\": container with ID starting with 2226fdee248bca22e7b63106c223dba65148c0bbd8b347f92c3322d5bb32a638 not found: ID does not exist" containerID="2226fdee248bca22e7b63106c223dba65148c0bbd8b347f92c3322d5bb32a638" Sep 09 02:13:17.915057 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.915004 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2226fdee248bca22e7b63106c223dba65148c0bbd8b347f92c3322d5bb32a638"} err="failed to get container status \"2226fdee248bca22e7b63106c223dba65148c0bbd8b347f92c3322d5bb32a638\": rpc error: code = NotFound desc = could not find container \"2226fdee248bca22e7b63106c223dba65148c0bbd8b347f92c3322d5bb32a638\": container with ID starting with 2226fdee248bca22e7b63106c223dba65148c0bbd8b347f92c3322d5bb32a638 not found: ID does not exist" Sep 09 02:13:17.915057 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.915039 2561 scope.go:117] "RemoveContainer" containerID="d2ac0c17949726b2ae1168c05b6b72a306c53679330bf3471e7dc9a9657310d4" Sep 09 02:13:17.915259 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:13:17.915243 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2ac0c17949726b2ae1168c05b6b72a306c53679330bf3471e7dc9a9657310d4\": container with ID starting with d2ac0c17949726b2ae1168c05b6b72a306c53679330bf3471e7dc9a9657310d4 not found: ID does not exist" containerID="d2ac0c17949726b2ae1168c05b6b72a306c53679330bf3471e7dc9a9657310d4" Sep 09 02:13:17.915301 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.915261 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2ac0c17949726b2ae1168c05b6b72a306c53679330bf3471e7dc9a9657310d4"} err="failed to get container status \"d2ac0c17949726b2ae1168c05b6b72a306c53679330bf3471e7dc9a9657310d4\": rpc error: code = NotFound desc = could not find container \"d2ac0c17949726b2ae1168c05b6b72a306c53679330bf3471e7dc9a9657310d4\": container with ID starting with d2ac0c17949726b2ae1168c05b6b72a306c53679330bf3471e7dc9a9657310d4 not found: ID does not exist" Sep 09 02:13:17.915301 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.915273 2561 scope.go:117] "RemoveContainer" containerID="56887768ecd4e1fbbbfa3428aa45d0adfae3bd39670faa1522cf0b0dbe9847df" Sep 09 02:13:17.915508 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:13:17.915489 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56887768ecd4e1fbbbfa3428aa45d0adfae3bd39670faa1522cf0b0dbe9847df\": container with ID starting with 56887768ecd4e1fbbbfa3428aa45d0adfae3bd39670faa1522cf0b0dbe9847df not found: ID does not exist" containerID="56887768ecd4e1fbbbfa3428aa45d0adfae3bd39670faa1522cf0b0dbe9847df" Sep 09 02:13:17.915553 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.915514 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56887768ecd4e1fbbbfa3428aa45d0adfae3bd39670faa1522cf0b0dbe9847df"} err="failed to get container status \"56887768ecd4e1fbbbfa3428aa45d0adfae3bd39670faa1522cf0b0dbe9847df\": rpc error: code = NotFound desc = could not find container \"56887768ecd4e1fbbbfa3428aa45d0adfae3bd39670faa1522cf0b0dbe9847df\": container with ID starting with 56887768ecd4e1fbbbfa3428aa45d0adfae3bd39670faa1522cf0b0dbe9847df not found: ID does not exist" Sep 09 02:13:17.915553 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.915530 2561 scope.go:117] "RemoveContainer" containerID="a731d9ba654e8291d9ec39a49510de31d2236cbadb1e259af6123a6e0c517bb3" Sep 09 02:13:17.915750 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:13:17.915731 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a731d9ba654e8291d9ec39a49510de31d2236cbadb1e259af6123a6e0c517bb3\": container with ID starting with a731d9ba654e8291d9ec39a49510de31d2236cbadb1e259af6123a6e0c517bb3 not found: ID does not exist" containerID="a731d9ba654e8291d9ec39a49510de31d2236cbadb1e259af6123a6e0c517bb3" Sep 09 02:13:17.915817 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.915758 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a731d9ba654e8291d9ec39a49510de31d2236cbadb1e259af6123a6e0c517bb3"} err="failed to get container status \"a731d9ba654e8291d9ec39a49510de31d2236cbadb1e259af6123a6e0c517bb3\": rpc error: code = NotFound desc = could not find container \"a731d9ba654e8291d9ec39a49510de31d2236cbadb1e259af6123a6e0c517bb3\": container with ID starting with a731d9ba654e8291d9ec39a49510de31d2236cbadb1e259af6123a6e0c517bb3 not found: ID does not exist" Sep 09 02:13:17.915817 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.915782 2561 scope.go:117] "RemoveContainer" containerID="99cfedc36e31101740f02272f980a801b0a48c269b0974b5ef7ea20f440bbd06" Sep 09 02:13:17.916007 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:13:17.915992 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99cfedc36e31101740f02272f980a801b0a48c269b0974b5ef7ea20f440bbd06\": container with ID starting with 99cfedc36e31101740f02272f980a801b0a48c269b0974b5ef7ea20f440bbd06 not found: ID does not exist" containerID="99cfedc36e31101740f02272f980a801b0a48c269b0974b5ef7ea20f440bbd06" Sep 09 02:13:17.916087 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.916024 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99cfedc36e31101740f02272f980a801b0a48c269b0974b5ef7ea20f440bbd06"} err="failed to get container status \"99cfedc36e31101740f02272f980a801b0a48c269b0974b5ef7ea20f440bbd06\": rpc error: code = NotFound desc = could not find container \"99cfedc36e31101740f02272f980a801b0a48c269b0974b5ef7ea20f440bbd06\": container with ID starting with 99cfedc36e31101740f02272f980a801b0a48c269b0974b5ef7ea20f440bbd06 not found: ID does not exist" Sep 09 02:13:17.916087 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.916039 2561 scope.go:117] "RemoveContainer" containerID="00be1a39065c82998b7ee3588fe21f26f342436e82a220f2cc344165521970cc" Sep 09 02:13:17.916282 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.916263 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00be1a39065c82998b7ee3588fe21f26f342436e82a220f2cc344165521970cc"} err="failed to get container status \"00be1a39065c82998b7ee3588fe21f26f342436e82a220f2cc344165521970cc\": rpc error: code = NotFound desc = could not find container \"00be1a39065c82998b7ee3588fe21f26f342436e82a220f2cc344165521970cc\": container with ID starting with 00be1a39065c82998b7ee3588fe21f26f342436e82a220f2cc344165521970cc not found: ID does not exist" Sep 09 02:13:17.916341 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.916284 2561 scope.go:117] "RemoveContainer" containerID="cb19d1ce2ae6e7eff7a5b969f5fdcd3578fa5a67c7a9a13f03265926f57fd9c2" Sep 09 02:13:17.916498 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.916482 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb19d1ce2ae6e7eff7a5b969f5fdcd3578fa5a67c7a9a13f03265926f57fd9c2"} err="failed to get container status \"cb19d1ce2ae6e7eff7a5b969f5fdcd3578fa5a67c7a9a13f03265926f57fd9c2\": rpc error: code = NotFound desc = could not find container \"cb19d1ce2ae6e7eff7a5b969f5fdcd3578fa5a67c7a9a13f03265926f57fd9c2\": container with ID starting with cb19d1ce2ae6e7eff7a5b969f5fdcd3578fa5a67c7a9a13f03265926f57fd9c2 not found: ID does not exist" Sep 09 02:13:17.916543 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.916499 2561 scope.go:117] "RemoveContainer" containerID="2226fdee248bca22e7b63106c223dba65148c0bbd8b347f92c3322d5bb32a638" Sep 09 02:13:17.916691 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.916673 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2226fdee248bca22e7b63106c223dba65148c0bbd8b347f92c3322d5bb32a638"} err="failed to get container status \"2226fdee248bca22e7b63106c223dba65148c0bbd8b347f92c3322d5bb32a638\": rpc error: code = NotFound desc = could not find container \"2226fdee248bca22e7b63106c223dba65148c0bbd8b347f92c3322d5bb32a638\": container with ID starting with 2226fdee248bca22e7b63106c223dba65148c0bbd8b347f92c3322d5bb32a638 not found: ID does not exist" Sep 09 02:13:17.916756 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.916692 2561 scope.go:117] "RemoveContainer" containerID="d2ac0c17949726b2ae1168c05b6b72a306c53679330bf3471e7dc9a9657310d4" Sep 09 02:13:17.916898 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.916882 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2ac0c17949726b2ae1168c05b6b72a306c53679330bf3471e7dc9a9657310d4"} err="failed to get container status \"d2ac0c17949726b2ae1168c05b6b72a306c53679330bf3471e7dc9a9657310d4\": rpc error: code = NotFound desc = could not find container \"d2ac0c17949726b2ae1168c05b6b72a306c53679330bf3471e7dc9a9657310d4\": container with ID starting with d2ac0c17949726b2ae1168c05b6b72a306c53679330bf3471e7dc9a9657310d4 not found: ID does not exist" Sep 09 02:13:17.916943 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.916898 2561 scope.go:117] "RemoveContainer" containerID="56887768ecd4e1fbbbfa3428aa45d0adfae3bd39670faa1522cf0b0dbe9847df" Sep 09 02:13:17.917161 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.917143 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56887768ecd4e1fbbbfa3428aa45d0adfae3bd39670faa1522cf0b0dbe9847df"} err="failed to get container status \"56887768ecd4e1fbbbfa3428aa45d0adfae3bd39670faa1522cf0b0dbe9847df\": rpc error: code = NotFound desc = could not find container \"56887768ecd4e1fbbbfa3428aa45d0adfae3bd39670faa1522cf0b0dbe9847df\": container with ID starting with 56887768ecd4e1fbbbfa3428aa45d0adfae3bd39670faa1522cf0b0dbe9847df not found: ID does not exist" Sep 09 02:13:17.917233 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.917162 2561 scope.go:117] "RemoveContainer" containerID="a731d9ba654e8291d9ec39a49510de31d2236cbadb1e259af6123a6e0c517bb3" Sep 09 02:13:17.917377 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.917360 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a731d9ba654e8291d9ec39a49510de31d2236cbadb1e259af6123a6e0c517bb3"} err="failed to get container status \"a731d9ba654e8291d9ec39a49510de31d2236cbadb1e259af6123a6e0c517bb3\": rpc error: code = NotFound desc = could not find container \"a731d9ba654e8291d9ec39a49510de31d2236cbadb1e259af6123a6e0c517bb3\": container with ID starting with a731d9ba654e8291d9ec39a49510de31d2236cbadb1e259af6123a6e0c517bb3 not found: ID does not exist" Sep 09 02:13:17.917425 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.917378 2561 scope.go:117] "RemoveContainer" containerID="99cfedc36e31101740f02272f980a801b0a48c269b0974b5ef7ea20f440bbd06" Sep 09 02:13:17.917565 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.917549 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99cfedc36e31101740f02272f980a801b0a48c269b0974b5ef7ea20f440bbd06"} err="failed to get container status \"99cfedc36e31101740f02272f980a801b0a48c269b0974b5ef7ea20f440bbd06\": rpc error: code = NotFound desc = could not find container \"99cfedc36e31101740f02272f980a801b0a48c269b0974b5ef7ea20f440bbd06\": container with ID starting with 99cfedc36e31101740f02272f980a801b0a48c269b0974b5ef7ea20f440bbd06 not found: ID does not exist" Sep 09 02:13:17.917638 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.917566 2561 scope.go:117] "RemoveContainer" containerID="00be1a39065c82998b7ee3588fe21f26f342436e82a220f2cc344165521970cc" Sep 09 02:13:17.917778 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.917762 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00be1a39065c82998b7ee3588fe21f26f342436e82a220f2cc344165521970cc"} err="failed to get container status \"00be1a39065c82998b7ee3588fe21f26f342436e82a220f2cc344165521970cc\": rpc error: code = NotFound desc = could not find container \"00be1a39065c82998b7ee3588fe21f26f342436e82a220f2cc344165521970cc\": container with ID starting with 00be1a39065c82998b7ee3588fe21f26f342436e82a220f2cc344165521970cc not found: ID does not exist" Sep 09 02:13:17.917822 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.917778 2561 scope.go:117] "RemoveContainer" containerID="cb19d1ce2ae6e7eff7a5b969f5fdcd3578fa5a67c7a9a13f03265926f57fd9c2" Sep 09 02:13:17.917964 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.917947 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb19d1ce2ae6e7eff7a5b969f5fdcd3578fa5a67c7a9a13f03265926f57fd9c2"} err="failed to get container status \"cb19d1ce2ae6e7eff7a5b969f5fdcd3578fa5a67c7a9a13f03265926f57fd9c2\": rpc error: code = NotFound desc = could not find container \"cb19d1ce2ae6e7eff7a5b969f5fdcd3578fa5a67c7a9a13f03265926f57fd9c2\": container with ID starting with cb19d1ce2ae6e7eff7a5b969f5fdcd3578fa5a67c7a9a13f03265926f57fd9c2 not found: ID does not exist" Sep 09 02:13:17.918047 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.917965 2561 scope.go:117] "RemoveContainer" containerID="2226fdee248bca22e7b63106c223dba65148c0bbd8b347f92c3322d5bb32a638" Sep 09 02:13:17.918181 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.918162 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2226fdee248bca22e7b63106c223dba65148c0bbd8b347f92c3322d5bb32a638"} err="failed to get container status \"2226fdee248bca22e7b63106c223dba65148c0bbd8b347f92c3322d5bb32a638\": rpc error: code = NotFound desc = could not find container \"2226fdee248bca22e7b63106c223dba65148c0bbd8b347f92c3322d5bb32a638\": container with ID starting with 2226fdee248bca22e7b63106c223dba65148c0bbd8b347f92c3322d5bb32a638 not found: ID does not exist" Sep 09 02:13:17.918248 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.918182 2561 scope.go:117] "RemoveContainer" containerID="d2ac0c17949726b2ae1168c05b6b72a306c53679330bf3471e7dc9a9657310d4" Sep 09 02:13:17.918382 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.918367 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2ac0c17949726b2ae1168c05b6b72a306c53679330bf3471e7dc9a9657310d4"} err="failed to get container status \"d2ac0c17949726b2ae1168c05b6b72a306c53679330bf3471e7dc9a9657310d4\": rpc error: code = NotFound desc = could not find container \"d2ac0c17949726b2ae1168c05b6b72a306c53679330bf3471e7dc9a9657310d4\": container with ID starting with d2ac0c17949726b2ae1168c05b6b72a306c53679330bf3471e7dc9a9657310d4 not found: ID does not exist" Sep 09 02:13:17.918427 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.918382 2561 scope.go:117] "RemoveContainer" containerID="56887768ecd4e1fbbbfa3428aa45d0adfae3bd39670faa1522cf0b0dbe9847df" Sep 09 02:13:17.918562 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.918545 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56887768ecd4e1fbbbfa3428aa45d0adfae3bd39670faa1522cf0b0dbe9847df"} err="failed to get container status \"56887768ecd4e1fbbbfa3428aa45d0adfae3bd39670faa1522cf0b0dbe9847df\": rpc error: code = NotFound desc = could not find container \"56887768ecd4e1fbbbfa3428aa45d0adfae3bd39670faa1522cf0b0dbe9847df\": container with ID starting with 56887768ecd4e1fbbbfa3428aa45d0adfae3bd39670faa1522cf0b0dbe9847df not found: ID does not exist" Sep 09 02:13:17.918627 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.918562 2561 scope.go:117] "RemoveContainer" containerID="a731d9ba654e8291d9ec39a49510de31d2236cbadb1e259af6123a6e0c517bb3" Sep 09 02:13:17.918770 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.918753 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a731d9ba654e8291d9ec39a49510de31d2236cbadb1e259af6123a6e0c517bb3"} err="failed to get container status \"a731d9ba654e8291d9ec39a49510de31d2236cbadb1e259af6123a6e0c517bb3\": rpc error: code = NotFound desc = could not find container \"a731d9ba654e8291d9ec39a49510de31d2236cbadb1e259af6123a6e0c517bb3\": container with ID starting with a731d9ba654e8291d9ec39a49510de31d2236cbadb1e259af6123a6e0c517bb3 not found: ID does not exist" Sep 09 02:13:17.918817 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.918770 2561 scope.go:117] "RemoveContainer" containerID="99cfedc36e31101740f02272f980a801b0a48c269b0974b5ef7ea20f440bbd06" Sep 09 02:13:17.918945 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.918931 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99cfedc36e31101740f02272f980a801b0a48c269b0974b5ef7ea20f440bbd06"} err="failed to get container status \"99cfedc36e31101740f02272f980a801b0a48c269b0974b5ef7ea20f440bbd06\": rpc error: code = NotFound desc = could not find container \"99cfedc36e31101740f02272f980a801b0a48c269b0974b5ef7ea20f440bbd06\": container with ID starting with 99cfedc36e31101740f02272f980a801b0a48c269b0974b5ef7ea20f440bbd06 not found: ID does not exist" Sep 09 02:13:17.918945 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.918944 2561 scope.go:117] "RemoveContainer" containerID="00be1a39065c82998b7ee3588fe21f26f342436e82a220f2cc344165521970cc" Sep 09 02:13:17.919119 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.919105 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00be1a39065c82998b7ee3588fe21f26f342436e82a220f2cc344165521970cc"} err="failed to get container status \"00be1a39065c82998b7ee3588fe21f26f342436e82a220f2cc344165521970cc\": rpc error: code = NotFound desc = could not find container \"00be1a39065c82998b7ee3588fe21f26f342436e82a220f2cc344165521970cc\": container with ID starting with 00be1a39065c82998b7ee3588fe21f26f342436e82a220f2cc344165521970cc not found: ID does not exist" Sep 09 02:13:17.919171 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.919121 2561 scope.go:117] "RemoveContainer" containerID="cb19d1ce2ae6e7eff7a5b969f5fdcd3578fa5a67c7a9a13f03265926f57fd9c2" Sep 09 02:13:17.919302 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.919286 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb19d1ce2ae6e7eff7a5b969f5fdcd3578fa5a67c7a9a13f03265926f57fd9c2"} err="failed to get container status \"cb19d1ce2ae6e7eff7a5b969f5fdcd3578fa5a67c7a9a13f03265926f57fd9c2\": rpc error: code = NotFound desc = could not find container \"cb19d1ce2ae6e7eff7a5b969f5fdcd3578fa5a67c7a9a13f03265926f57fd9c2\": container with ID starting with cb19d1ce2ae6e7eff7a5b969f5fdcd3578fa5a67c7a9a13f03265926f57fd9c2 not found: ID does not exist" Sep 09 02:13:17.919350 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.919301 2561 scope.go:117] "RemoveContainer" containerID="2226fdee248bca22e7b63106c223dba65148c0bbd8b347f92c3322d5bb32a638" Sep 09 02:13:17.919524 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.919507 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2226fdee248bca22e7b63106c223dba65148c0bbd8b347f92c3322d5bb32a638"} err="failed to get container status \"2226fdee248bca22e7b63106c223dba65148c0bbd8b347f92c3322d5bb32a638\": rpc error: code = NotFound desc = could not find container \"2226fdee248bca22e7b63106c223dba65148c0bbd8b347f92c3322d5bb32a638\": container with ID starting with 2226fdee248bca22e7b63106c223dba65148c0bbd8b347f92c3322d5bb32a638 not found: ID does not exist" Sep 09 02:13:17.919524 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.919522 2561 scope.go:117] "RemoveContainer" containerID="d2ac0c17949726b2ae1168c05b6b72a306c53679330bf3471e7dc9a9657310d4" Sep 09 02:13:17.919704 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.919689 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2ac0c17949726b2ae1168c05b6b72a306c53679330bf3471e7dc9a9657310d4"} err="failed to get container status \"d2ac0c17949726b2ae1168c05b6b72a306c53679330bf3471e7dc9a9657310d4\": rpc error: code = NotFound desc = could not find container \"d2ac0c17949726b2ae1168c05b6b72a306c53679330bf3471e7dc9a9657310d4\": container with ID starting with d2ac0c17949726b2ae1168c05b6b72a306c53679330bf3471e7dc9a9657310d4 not found: ID does not exist" Sep 09 02:13:17.919704 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.919703 2561 scope.go:117] "RemoveContainer" containerID="56887768ecd4e1fbbbfa3428aa45d0adfae3bd39670faa1522cf0b0dbe9847df" Sep 09 02:13:17.919875 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.919861 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56887768ecd4e1fbbbfa3428aa45d0adfae3bd39670faa1522cf0b0dbe9847df"} err="failed to get container status \"56887768ecd4e1fbbbfa3428aa45d0adfae3bd39670faa1522cf0b0dbe9847df\": rpc error: code = NotFound desc = could not find container \"56887768ecd4e1fbbbfa3428aa45d0adfae3bd39670faa1522cf0b0dbe9847df\": container with ID starting with 56887768ecd4e1fbbbfa3428aa45d0adfae3bd39670faa1522cf0b0dbe9847df not found: ID does not exist" Sep 09 02:13:17.919922 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.919875 2561 scope.go:117] "RemoveContainer" containerID="a731d9ba654e8291d9ec39a49510de31d2236cbadb1e259af6123a6e0c517bb3" Sep 09 02:13:17.920112 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.920097 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a731d9ba654e8291d9ec39a49510de31d2236cbadb1e259af6123a6e0c517bb3"} err="failed to get container status \"a731d9ba654e8291d9ec39a49510de31d2236cbadb1e259af6123a6e0c517bb3\": rpc error: code = NotFound desc = could not find container \"a731d9ba654e8291d9ec39a49510de31d2236cbadb1e259af6123a6e0c517bb3\": container with ID starting with a731d9ba654e8291d9ec39a49510de31d2236cbadb1e259af6123a6e0c517bb3 not found: ID does not exist" Sep 09 02:13:17.920112 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.920111 2561 scope.go:117] "RemoveContainer" containerID="99cfedc36e31101740f02272f980a801b0a48c269b0974b5ef7ea20f440bbd06" Sep 09 02:13:17.920330 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.920308 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99cfedc36e31101740f02272f980a801b0a48c269b0974b5ef7ea20f440bbd06"} err="failed to get container status \"99cfedc36e31101740f02272f980a801b0a48c269b0974b5ef7ea20f440bbd06\": rpc error: code = NotFound desc = could not find container \"99cfedc36e31101740f02272f980a801b0a48c269b0974b5ef7ea20f440bbd06\": container with ID starting with 99cfedc36e31101740f02272f980a801b0a48c269b0974b5ef7ea20f440bbd06 not found: ID does not exist" Sep 09 02:13:17.920372 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.920331 2561 scope.go:117] "RemoveContainer" containerID="00be1a39065c82998b7ee3588fe21f26f342436e82a220f2cc344165521970cc" Sep 09 02:13:17.920530 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.920511 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00be1a39065c82998b7ee3588fe21f26f342436e82a220f2cc344165521970cc"} err="failed to get container status \"00be1a39065c82998b7ee3588fe21f26f342436e82a220f2cc344165521970cc\": rpc error: code = NotFound desc = could not find container \"00be1a39065c82998b7ee3588fe21f26f342436e82a220f2cc344165521970cc\": container with ID starting with 00be1a39065c82998b7ee3588fe21f26f342436e82a220f2cc344165521970cc not found: ID does not exist" Sep 09 02:13:17.920594 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.920532 2561 scope.go:117] "RemoveContainer" containerID="cb19d1ce2ae6e7eff7a5b969f5fdcd3578fa5a67c7a9a13f03265926f57fd9c2" Sep 09 02:13:17.920760 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.920743 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb19d1ce2ae6e7eff7a5b969f5fdcd3578fa5a67c7a9a13f03265926f57fd9c2"} err="failed to get container status \"cb19d1ce2ae6e7eff7a5b969f5fdcd3578fa5a67c7a9a13f03265926f57fd9c2\": rpc error: code = NotFound desc = could not find container \"cb19d1ce2ae6e7eff7a5b969f5fdcd3578fa5a67c7a9a13f03265926f57fd9c2\": container with ID starting with cb19d1ce2ae6e7eff7a5b969f5fdcd3578fa5a67c7a9a13f03265926f57fd9c2 not found: ID does not exist" Sep 09 02:13:17.920808 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.920760 2561 scope.go:117] "RemoveContainer" containerID="2226fdee248bca22e7b63106c223dba65148c0bbd8b347f92c3322d5bb32a638" Sep 09 02:13:17.920989 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.920972 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2226fdee248bca22e7b63106c223dba65148c0bbd8b347f92c3322d5bb32a638"} err="failed to get container status \"2226fdee248bca22e7b63106c223dba65148c0bbd8b347f92c3322d5bb32a638\": rpc error: code = NotFound desc = could not find container \"2226fdee248bca22e7b63106c223dba65148c0bbd8b347f92c3322d5bb32a638\": container with ID starting with 2226fdee248bca22e7b63106c223dba65148c0bbd8b347f92c3322d5bb32a638 not found: ID does not exist" Sep 09 02:13:17.921080 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.920991 2561 scope.go:117] "RemoveContainer" containerID="d2ac0c17949726b2ae1168c05b6b72a306c53679330bf3471e7dc9a9657310d4" Sep 09 02:13:17.921238 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.921217 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2ac0c17949726b2ae1168c05b6b72a306c53679330bf3471e7dc9a9657310d4"} err="failed to get container status \"d2ac0c17949726b2ae1168c05b6b72a306c53679330bf3471e7dc9a9657310d4\": rpc error: code = NotFound desc = could not find container \"d2ac0c17949726b2ae1168c05b6b72a306c53679330bf3471e7dc9a9657310d4\": container with ID starting with d2ac0c17949726b2ae1168c05b6b72a306c53679330bf3471e7dc9a9657310d4 not found: ID does not exist" Sep 09 02:13:17.921534 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.921238 2561 scope.go:117] "RemoveContainer" containerID="56887768ecd4e1fbbbfa3428aa45d0adfae3bd39670faa1522cf0b0dbe9847df" Sep 09 02:13:17.921534 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.921444 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56887768ecd4e1fbbbfa3428aa45d0adfae3bd39670faa1522cf0b0dbe9847df"} err="failed to get container status \"56887768ecd4e1fbbbfa3428aa45d0adfae3bd39670faa1522cf0b0dbe9847df\": rpc error: code = NotFound desc = could not find container \"56887768ecd4e1fbbbfa3428aa45d0adfae3bd39670faa1522cf0b0dbe9847df\": container with ID starting with 56887768ecd4e1fbbbfa3428aa45d0adfae3bd39670faa1522cf0b0dbe9847df not found: ID does not exist" Sep 09 02:13:17.921534 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.921465 2561 scope.go:117] "RemoveContainer" containerID="a731d9ba654e8291d9ec39a49510de31d2236cbadb1e259af6123a6e0c517bb3" Sep 09 02:13:17.921707 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.921690 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a731d9ba654e8291d9ec39a49510de31d2236cbadb1e259af6123a6e0c517bb3"} err="failed to get container status \"a731d9ba654e8291d9ec39a49510de31d2236cbadb1e259af6123a6e0c517bb3\": rpc error: code = NotFound desc = could not find container \"a731d9ba654e8291d9ec39a49510de31d2236cbadb1e259af6123a6e0c517bb3\": container with ID starting with a731d9ba654e8291d9ec39a49510de31d2236cbadb1e259af6123a6e0c517bb3 not found: ID does not exist" Sep 09 02:13:17.921752 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.921707 2561 scope.go:117] "RemoveContainer" containerID="99cfedc36e31101740f02272f980a801b0a48c269b0974b5ef7ea20f440bbd06" Sep 09 02:13:17.921906 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.921888 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99cfedc36e31101740f02272f980a801b0a48c269b0974b5ef7ea20f440bbd06"} err="failed to get container status \"99cfedc36e31101740f02272f980a801b0a48c269b0974b5ef7ea20f440bbd06\": rpc error: code = NotFound desc = could not find container \"99cfedc36e31101740f02272f980a801b0a48c269b0974b5ef7ea20f440bbd06\": container with ID starting with 99cfedc36e31101740f02272f980a801b0a48c269b0974b5ef7ea20f440bbd06 not found: ID does not exist" Sep 09 02:13:17.921968 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.921907 2561 scope.go:117] "RemoveContainer" containerID="00be1a39065c82998b7ee3588fe21f26f342436e82a220f2cc344165521970cc" Sep 09 02:13:17.922125 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.922109 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00be1a39065c82998b7ee3588fe21f26f342436e82a220f2cc344165521970cc"} err="failed to get container status \"00be1a39065c82998b7ee3588fe21f26f342436e82a220f2cc344165521970cc\": rpc error: code = NotFound desc = could not find container \"00be1a39065c82998b7ee3588fe21f26f342436e82a220f2cc344165521970cc\": container with ID starting with 00be1a39065c82998b7ee3588fe21f26f342436e82a220f2cc344165521970cc not found: ID does not exist" Sep 09 02:13:17.922178 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.922127 2561 scope.go:117] "RemoveContainer" containerID="cb19d1ce2ae6e7eff7a5b969f5fdcd3578fa5a67c7a9a13f03265926f57fd9c2" Sep 09 02:13:17.922334 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.922316 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb19d1ce2ae6e7eff7a5b969f5fdcd3578fa5a67c7a9a13f03265926f57fd9c2"} err="failed to get container status \"cb19d1ce2ae6e7eff7a5b969f5fdcd3578fa5a67c7a9a13f03265926f57fd9c2\": rpc error: code = NotFound desc = could not find container \"cb19d1ce2ae6e7eff7a5b969f5fdcd3578fa5a67c7a9a13f03265926f57fd9c2\": container with ID starting with cb19d1ce2ae6e7eff7a5b969f5fdcd3578fa5a67c7a9a13f03265926f57fd9c2 not found: ID does not exist" Sep 09 02:13:17.922401 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.922334 2561 scope.go:117] "RemoveContainer" containerID="2226fdee248bca22e7b63106c223dba65148c0bbd8b347f92c3322d5bb32a638" Sep 09 02:13:17.922550 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.922534 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2226fdee248bca22e7b63106c223dba65148c0bbd8b347f92c3322d5bb32a638"} err="failed to get container status \"2226fdee248bca22e7b63106c223dba65148c0bbd8b347f92c3322d5bb32a638\": rpc error: code = NotFound desc = could not find container \"2226fdee248bca22e7b63106c223dba65148c0bbd8b347f92c3322d5bb32a638\": container with ID starting with 2226fdee248bca22e7b63106c223dba65148c0bbd8b347f92c3322d5bb32a638 not found: ID does not exist" Sep 09 02:13:17.922598 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.922550 2561 scope.go:117] "RemoveContainer" containerID="d2ac0c17949726b2ae1168c05b6b72a306c53679330bf3471e7dc9a9657310d4" Sep 09 02:13:17.922733 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.922716 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2ac0c17949726b2ae1168c05b6b72a306c53679330bf3471e7dc9a9657310d4"} err="failed to get container status \"d2ac0c17949726b2ae1168c05b6b72a306c53679330bf3471e7dc9a9657310d4\": rpc error: code = NotFound desc = could not find container \"d2ac0c17949726b2ae1168c05b6b72a306c53679330bf3471e7dc9a9657310d4\": container with ID starting with d2ac0c17949726b2ae1168c05b6b72a306c53679330bf3471e7dc9a9657310d4 not found: ID does not exist" Sep 09 02:13:17.922798 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.922734 2561 scope.go:117] "RemoveContainer" containerID="56887768ecd4e1fbbbfa3428aa45d0adfae3bd39670faa1522cf0b0dbe9847df" Sep 09 02:13:17.922941 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.922927 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56887768ecd4e1fbbbfa3428aa45d0adfae3bd39670faa1522cf0b0dbe9847df"} err="failed to get container status \"56887768ecd4e1fbbbfa3428aa45d0adfae3bd39670faa1522cf0b0dbe9847df\": rpc error: code = NotFound desc = could not find container \"56887768ecd4e1fbbbfa3428aa45d0adfae3bd39670faa1522cf0b0dbe9847df\": container with ID starting with 56887768ecd4e1fbbbfa3428aa45d0adfae3bd39670faa1522cf0b0dbe9847df not found: ID does not exist" Sep 09 02:13:17.922995 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.922941 2561 scope.go:117] "RemoveContainer" containerID="a731d9ba654e8291d9ec39a49510de31d2236cbadb1e259af6123a6e0c517bb3" Sep 09 02:13:17.923215 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.923190 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a731d9ba654e8291d9ec39a49510de31d2236cbadb1e259af6123a6e0c517bb3"} err="failed to get container status \"a731d9ba654e8291d9ec39a49510de31d2236cbadb1e259af6123a6e0c517bb3\": rpc error: code = NotFound desc = could not find container \"a731d9ba654e8291d9ec39a49510de31d2236cbadb1e259af6123a6e0c517bb3\": container with ID starting with a731d9ba654e8291d9ec39a49510de31d2236cbadb1e259af6123a6e0c517bb3 not found: ID does not exist" Sep 09 02:13:17.923303 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.923219 2561 scope.go:117] "RemoveContainer" containerID="99cfedc36e31101740f02272f980a801b0a48c269b0974b5ef7ea20f440bbd06" Sep 09 02:13:17.924055 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.923537 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99cfedc36e31101740f02272f980a801b0a48c269b0974b5ef7ea20f440bbd06"} err="failed to get container status \"99cfedc36e31101740f02272f980a801b0a48c269b0974b5ef7ea20f440bbd06\": rpc error: code = NotFound desc = could not find container \"99cfedc36e31101740f02272f980a801b0a48c269b0974b5ef7ea20f440bbd06\": container with ID starting with 99cfedc36e31101740f02272f980a801b0a48c269b0974b5ef7ea20f440bbd06 not found: ID does not exist" Sep 09 02:13:17.925270 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.925251 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 02:13:17.925547 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.925533 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="651a2e91-648c-465e-8726-94efaff2bbf2" containerName="init-config-reloader" Sep 09 02:13:17.925611 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.925551 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="651a2e91-648c-465e-8726-94efaff2bbf2" containerName="init-config-reloader" Sep 09 02:13:17.925611 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.925561 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="651a2e91-648c-465e-8726-94efaff2bbf2" containerName="thanos-sidecar" Sep 09 02:13:17.925611 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.925569 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="651a2e91-648c-465e-8726-94efaff2bbf2" containerName="thanos-sidecar" Sep 09 02:13:17.925611 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.925579 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="651a2e91-648c-465e-8726-94efaff2bbf2" containerName="kube-rbac-proxy" Sep 09 02:13:17.925611 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.925587 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="651a2e91-648c-465e-8726-94efaff2bbf2" containerName="kube-rbac-proxy" Sep 09 02:13:17.925753 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.925612 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="651a2e91-648c-465e-8726-94efaff2bbf2" containerName="config-reloader" Sep 09 02:13:17.925753 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.925620 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="651a2e91-648c-465e-8726-94efaff2bbf2" containerName="config-reloader" Sep 09 02:13:17.925753 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.925631 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="651a2e91-648c-465e-8726-94efaff2bbf2" containerName="prometheus" Sep 09 02:13:17.925753 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.925638 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="651a2e91-648c-465e-8726-94efaff2bbf2" containerName="prometheus" Sep 09 02:13:17.925753 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.925650 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="651a2e91-648c-465e-8726-94efaff2bbf2" containerName="kube-rbac-proxy-web" Sep 09 02:13:17.925753 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.925657 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="651a2e91-648c-465e-8726-94efaff2bbf2" containerName="kube-rbac-proxy-web" Sep 09 02:13:17.925753 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.925665 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="651a2e91-648c-465e-8726-94efaff2bbf2" containerName="kube-rbac-proxy-thanos" Sep 09 02:13:17.925753 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.925672 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="651a2e91-648c-465e-8726-94efaff2bbf2" containerName="kube-rbac-proxy-thanos" Sep 09 02:13:17.925753 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.925743 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="651a2e91-648c-465e-8726-94efaff2bbf2" containerName="prometheus" Sep 09 02:13:17.925753 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.925752 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="651a2e91-648c-465e-8726-94efaff2bbf2" containerName="kube-rbac-proxy-thanos" Sep 09 02:13:17.926044 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.925762 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="651a2e91-648c-465e-8726-94efaff2bbf2" containerName="config-reloader" Sep 09 02:13:17.926044 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.925769 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="651a2e91-648c-465e-8726-94efaff2bbf2" containerName="thanos-sidecar" Sep 09 02:13:17.926044 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.925777 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="651a2e91-648c-465e-8726-94efaff2bbf2" containerName="kube-rbac-proxy" Sep 09 02:13:17.926044 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.925786 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="651a2e91-648c-465e-8726-94efaff2bbf2" containerName="kube-rbac-proxy-web" Sep 09 02:13:17.927919 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.927906 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:17.930386 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.930350 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 09 02:13:17.930522 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.930504 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 09 02:13:17.930673 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.930657 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 09 02:13:17.930747 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.930727 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-9q75ht698hars\"" Sep 09 02:13:17.931001 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.930972 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 09 02:13:17.931275 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.931256 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 09 02:13:17.931575 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.931558 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 09 02:13:17.931637 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.931608 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 09 02:13:17.931858 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.931838 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 09 02:13:17.931962 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.931867 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 09 02:13:17.932133 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.932117 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 09 02:13:17.932206 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.932141 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 09 02:13:17.932206 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.932151 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-s7x67\"" Sep 09 02:13:17.932206 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.932193 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 09 02:13:17.932439 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.932423 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 09 02:13:17.935625 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.935504 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 09 02:13:17.937815 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.937791 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 09 02:13:17.954518 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.954493 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 02:13:17.993042 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.992980 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8e358331-a769-4146-a574-f9642e8e9705-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:17.993042 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.993044 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8e358331-a769-4146-a574-f9642e8e9705-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:17.993284 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.993094 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8e358331-a769-4146-a574-f9642e8e9705-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:17.993284 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.993135 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8e358331-a769-4146-a574-f9642e8e9705-config-out\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:17.993284 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.993154 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8e358331-a769-4146-a574-f9642e8e9705-web-config\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:17.993284 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.993186 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8e358331-a769-4146-a574-f9642e8e9705-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:17.993284 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.993209 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8e358331-a769-4146-a574-f9642e8e9705-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:17.993284 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.993252 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8e358331-a769-4146-a574-f9642e8e9705-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:17.993494 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.993286 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8e358331-a769-4146-a574-f9642e8e9705-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:17.993494 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.993308 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e358331-a769-4146-a574-f9642e8e9705-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:17.993494 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.993328 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8e358331-a769-4146-a574-f9642e8e9705-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:17.993494 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.993366 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8e358331-a769-4146-a574-f9642e8e9705-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:17.993494 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.993406 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nzpn\" (UniqueName: \"kubernetes.io/projected/8e358331-a769-4146-a574-f9642e8e9705-kube-api-access-7nzpn\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:17.993494 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.993433 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8e358331-a769-4146-a574-f9642e8e9705-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:17.993494 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.993455 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8e358331-a769-4146-a574-f9642e8e9705-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:17.993494 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.993486 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e358331-a769-4146-a574-f9642e8e9705-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:17.993766 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.993510 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e358331-a769-4146-a574-f9642e8e9705-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:17.993766 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.993539 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8e358331-a769-4146-a574-f9642e8e9705-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:17.993766 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.993558 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8e358331-a769-4146-a574-f9642e8e9705-config\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:17.993766 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:17.993574 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e358331-a769-4146-a574-f9642e8e9705-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.094950 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.094919 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8e358331-a769-4146-a574-f9642e8e9705-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.095053 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.094955 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8e358331-a769-4146-a574-f9642e8e9705-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.095053 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.094972 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e358331-a769-4146-a574-f9642e8e9705-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.095136 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.095087 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e358331-a769-4146-a574-f9642e8e9705-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.095136 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.095126 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8e358331-a769-4146-a574-f9642e8e9705-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.095392 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.095251 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8e358331-a769-4146-a574-f9642e8e9705-config\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.095392 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.095296 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e358331-a769-4146-a574-f9642e8e9705-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.095392 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.095349 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8e358331-a769-4146-a574-f9642e8e9705-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.095392 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.095375 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8e358331-a769-4146-a574-f9642e8e9705-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.095657 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.095402 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8e358331-a769-4146-a574-f9642e8e9705-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.095657 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.095429 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8e358331-a769-4146-a574-f9642e8e9705-config-out\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.095657 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.095449 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8e358331-a769-4146-a574-f9642e8e9705-web-config\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.095657 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.095488 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8e358331-a769-4146-a574-f9642e8e9705-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.095657 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.095512 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8e358331-a769-4146-a574-f9642e8e9705-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.095657 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.095538 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8e358331-a769-4146-a574-f9642e8e9705-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.095657 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.095563 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8e358331-a769-4146-a574-f9642e8e9705-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.095657 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.095595 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e358331-a769-4146-a574-f9642e8e9705-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.095657 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.095626 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8e358331-a769-4146-a574-f9642e8e9705-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.095657 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.095656 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8e358331-a769-4146-a574-f9642e8e9705-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.096165 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.095697 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nzpn\" (UniqueName: \"kubernetes.io/projected/8e358331-a769-4146-a574-f9642e8e9705-kube-api-access-7nzpn\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.096165 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.095868 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e358331-a769-4146-a574-f9642e8e9705-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.096165 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.095987 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e358331-a769-4146-a574-f9642e8e9705-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.096165 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.096036 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e358331-a769-4146-a574-f9642e8e9705-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.096758 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.096350 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8e358331-a769-4146-a574-f9642e8e9705-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.096758 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.096395 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8e358331-a769-4146-a574-f9642e8e9705-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.097677 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.097650 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8e358331-a769-4146-a574-f9642e8e9705-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.098750 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.098309 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e358331-a769-4146-a574-f9642e8e9705-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.098750 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.098559 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8e358331-a769-4146-a574-f9642e8e9705-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.098750 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.098700 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/8e358331-a769-4146-a574-f9642e8e9705-config\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.098961 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.098930 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8e358331-a769-4146-a574-f9642e8e9705-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.099438 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.099414 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8e358331-a769-4146-a574-f9642e8e9705-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.099665 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.099645 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8e358331-a769-4146-a574-f9642e8e9705-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.099749 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.099665 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8e358331-a769-4146-a574-f9642e8e9705-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.099952 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.099930 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8e358331-a769-4146-a574-f9642e8e9705-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.100037 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.099959 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8e358331-a769-4146-a574-f9642e8e9705-web-config\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.100505 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.100487 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8e358331-a769-4146-a574-f9642e8e9705-config-out\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.100989 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.100967 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8e358331-a769-4146-a574-f9642e8e9705-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.101130 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.101112 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8e358331-a769-4146-a574-f9642e8e9705-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.101706 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.101690 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8e358331-a769-4146-a574-f9642e8e9705-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.104820 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.104802 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nzpn\" (UniqueName: \"kubernetes.io/projected/8e358331-a769-4146-a574-f9642e8e9705-kube-api-access-7nzpn\") pod \"prometheus-k8s-0\" (UID: \"8e358331-a769-4146-a574-f9642e8e9705\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.250322 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.250271 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:13:18.532876 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.532778 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 02:13:18.537259 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:13:18.537231 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e358331_a769_4146_a574_f9642e8e9705.slice/crio-43127ae058e5fe3c47b6e98c8ded0bccd2a05d2af713ba899b5ad1a85c7309d1 WatchSource:0}: Error finding container 43127ae058e5fe3c47b6e98c8ded0bccd2a05d2af713ba899b5ad1a85c7309d1: Status 404 returned error can't find the container with id 43127ae058e5fe3c47b6e98c8ded0bccd2a05d2af713ba899b5ad1a85c7309d1 Sep 09 02:13:18.869697 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.869604 2561 generic.go:356] "Generic (PLEG): container finished" podID="8e358331-a769-4146-a574-f9642e8e9705" containerID="a5b82671b9d812effa21657dc37102e8cc13146a034276b7a093abc4f6823574" exitCode=0 Sep 09 02:13:18.869697 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.869672 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8e358331-a769-4146-a574-f9642e8e9705","Type":"ContainerDied","Data":"a5b82671b9d812effa21657dc37102e8cc13146a034276b7a093abc4f6823574"} Sep 09 02:13:18.869885 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:18.869739 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8e358331-a769-4146-a574-f9642e8e9705","Type":"ContainerStarted","Data":"43127ae058e5fe3c47b6e98c8ded0bccd2a05d2af713ba899b5ad1a85c7309d1"} Sep 09 02:13:19.111611 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:19.111206 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="651a2e91-648c-465e-8726-94efaff2bbf2" path="/var/lib/kubelet/pods/651a2e91-648c-465e-8726-94efaff2bbf2/volumes" Sep 09 02:13:19.876561 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:19.876479 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8e358331-a769-4146-a574-f9642e8e9705","Type":"ContainerStarted","Data":"5106a744c1ba046d08dbd71860f393ba8384ffb07402d45d91722c102e4d9942"} Sep 09 02:13:19.876561 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:19.876543 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8e358331-a769-4146-a574-f9642e8e9705","Type":"ContainerStarted","Data":"b2695179eb26bd5b97373055588b4b50334d77c500b90d8e8e2dbbda3bed4a94"} Sep 09 02:13:19.876740 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:19.876568 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8e358331-a769-4146-a574-f9642e8e9705","Type":"ContainerStarted","Data":"6542bc7a4d5770a4079f084fd255e40c7acc3b336231d3e31f2eff42fa29eea4"} Sep 09 02:13:19.876740 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:19.876595 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8e358331-a769-4146-a574-f9642e8e9705","Type":"ContainerStarted","Data":"5985ed6bc25be15ecc0e7e75c3352944557b3958b9049ad433a45f4c247e73e2"} Sep 09 02:13:19.876740 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:19.876618 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8e358331-a769-4146-a574-f9642e8e9705","Type":"ContainerStarted","Data":"dfa0dcf5ef4eb71670920061411cc8b15b57fe619860a16ce74fa622db004023"} Sep 09 02:13:19.876740 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:19.876640 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8e358331-a769-4146-a574-f9642e8e9705","Type":"ContainerStarted","Data":"9a8e0859d4344cbd6995d4be1390e8c4b8282b63a5ce76ddeb02c2a5bf23b226"} Sep 09 02:13:19.906695 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:19.906642 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.886677588 podStartE2EDuration="2.906626189s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.948601ms" isStatefulPod=false podCreationTimestamp="2026-09-09 02:13:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:13:19.905804173 +0000 UTC m=+159.280835584" watchObservedRunningTime="2026-09-09 02:13:19.906626189 +0000 UTC m=+159.281657597" Sep 09 02:13:23.251323 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:13:23.251281 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:14:18.250729 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:14:18.250689 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:14:18.265918 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:14:18.265889 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:14:19.085512 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:14:19.085484 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 02:15:27.727892 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:27.727856 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b45446f48-4mb2l"] Sep 09 02:15:27.731081 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:27.731059 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-b45446f48-4mb2l" Sep 09 02:15:27.733312 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:27.733292 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 09 02:15:27.733312 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:27.733303 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-nhsqr\"" Sep 09 02:15:27.733525 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:27.733503 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 09 02:15:27.733525 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:27.733517 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 09 02:15:27.733629 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:27.733542 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 09 02:15:27.752743 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:27.752718 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b45446f48-4mb2l"] Sep 09 02:15:27.778494 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:27.778468 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6695767e-4dfb-4603-8e8c-b410fe2c61c9-webhook-cert\") pod \"opendatahub-operator-controller-manager-b45446f48-4mb2l\" (UID: \"6695767e-4dfb-4603-8e8c-b410fe2c61c9\") " pod="opendatahub/opendatahub-operator-controller-manager-b45446f48-4mb2l" Sep 09 02:15:27.778635 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:27.778514 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6695767e-4dfb-4603-8e8c-b410fe2c61c9-manifests\") pod \"opendatahub-operator-controller-manager-b45446f48-4mb2l\" (UID: \"6695767e-4dfb-4603-8e8c-b410fe2c61c9\") " pod="opendatahub/opendatahub-operator-controller-manager-b45446f48-4mb2l" Sep 09 02:15:27.778635 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:27.778542 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6695767e-4dfb-4603-8e8c-b410fe2c61c9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b45446f48-4mb2l\" (UID: \"6695767e-4dfb-4603-8e8c-b410fe2c61c9\") " pod="opendatahub/opendatahub-operator-controller-manager-b45446f48-4mb2l" Sep 09 02:15:27.778635 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:27.778585 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zvfk\" (UniqueName: \"kubernetes.io/projected/6695767e-4dfb-4603-8e8c-b410fe2c61c9-kube-api-access-9zvfk\") pod \"opendatahub-operator-controller-manager-b45446f48-4mb2l\" (UID: \"6695767e-4dfb-4603-8e8c-b410fe2c61c9\") " pod="opendatahub/opendatahub-operator-controller-manager-b45446f48-4mb2l" Sep 09 02:15:27.778750 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:27.778673 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6695767e-4dfb-4603-8e8c-b410fe2c61c9-tmp\") pod \"opendatahub-operator-controller-manager-b45446f48-4mb2l\" (UID: \"6695767e-4dfb-4603-8e8c-b410fe2c61c9\") " pod="opendatahub/opendatahub-operator-controller-manager-b45446f48-4mb2l" Sep 09 02:15:27.879499 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:27.879460 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6695767e-4dfb-4603-8e8c-b410fe2c61c9-tmp\") pod \"opendatahub-operator-controller-manager-b45446f48-4mb2l\" (UID: \"6695767e-4dfb-4603-8e8c-b410fe2c61c9\") " pod="opendatahub/opendatahub-operator-controller-manager-b45446f48-4mb2l" Sep 09 02:15:27.879668 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:27.879512 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6695767e-4dfb-4603-8e8c-b410fe2c61c9-webhook-cert\") pod \"opendatahub-operator-controller-manager-b45446f48-4mb2l\" (UID: \"6695767e-4dfb-4603-8e8c-b410fe2c61c9\") " pod="opendatahub/opendatahub-operator-controller-manager-b45446f48-4mb2l" Sep 09 02:15:27.879668 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:27.879538 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6695767e-4dfb-4603-8e8c-b410fe2c61c9-manifests\") pod \"opendatahub-operator-controller-manager-b45446f48-4mb2l\" (UID: \"6695767e-4dfb-4603-8e8c-b410fe2c61c9\") " pod="opendatahub/opendatahub-operator-controller-manager-b45446f48-4mb2l" Sep 09 02:15:27.879668 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:27.879562 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6695767e-4dfb-4603-8e8c-b410fe2c61c9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b45446f48-4mb2l\" (UID: \"6695767e-4dfb-4603-8e8c-b410fe2c61c9\") " pod="opendatahub/opendatahub-operator-controller-manager-b45446f48-4mb2l" Sep 09 02:15:27.879668 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:27.879578 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zvfk\" (UniqueName: \"kubernetes.io/projected/6695767e-4dfb-4603-8e8c-b410fe2c61c9-kube-api-access-9zvfk\") pod \"opendatahub-operator-controller-manager-b45446f48-4mb2l\" (UID: \"6695767e-4dfb-4603-8e8c-b410fe2c61c9\") " pod="opendatahub/opendatahub-operator-controller-manager-b45446f48-4mb2l" Sep 09 02:15:27.880039 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:27.879980 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6695767e-4dfb-4603-8e8c-b410fe2c61c9-manifests\") pod \"opendatahub-operator-controller-manager-b45446f48-4mb2l\" (UID: \"6695767e-4dfb-4603-8e8c-b410fe2c61c9\") " pod="opendatahub/opendatahub-operator-controller-manager-b45446f48-4mb2l" Sep 09 02:15:27.881914 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:27.881889 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6695767e-4dfb-4603-8e8c-b410fe2c61c9-tmp\") pod \"opendatahub-operator-controller-manager-b45446f48-4mb2l\" (UID: \"6695767e-4dfb-4603-8e8c-b410fe2c61c9\") " pod="opendatahub/opendatahub-operator-controller-manager-b45446f48-4mb2l" Sep 09 02:15:27.882052 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:27.882032 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6695767e-4dfb-4603-8e8c-b410fe2c61c9-webhook-cert\") pod \"opendatahub-operator-controller-manager-b45446f48-4mb2l\" (UID: \"6695767e-4dfb-4603-8e8c-b410fe2c61c9\") " pod="opendatahub/opendatahub-operator-controller-manager-b45446f48-4mb2l" Sep 09 02:15:27.882153 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:27.882135 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6695767e-4dfb-4603-8e8c-b410fe2c61c9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b45446f48-4mb2l\" (UID: \"6695767e-4dfb-4603-8e8c-b410fe2c61c9\") " pod="opendatahub/opendatahub-operator-controller-manager-b45446f48-4mb2l" Sep 09 02:15:27.889315 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:27.889291 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zvfk\" (UniqueName: \"kubernetes.io/projected/6695767e-4dfb-4603-8e8c-b410fe2c61c9-kube-api-access-9zvfk\") pod \"opendatahub-operator-controller-manager-b45446f48-4mb2l\" (UID: \"6695767e-4dfb-4603-8e8c-b410fe2c61c9\") " pod="opendatahub/opendatahub-operator-controller-manager-b45446f48-4mb2l" Sep 09 02:15:28.041479 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:28.041385 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-b45446f48-4mb2l" Sep 09 02:15:28.316365 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:28.316280 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b45446f48-4mb2l"] Sep 09 02:15:28.319294 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:15:28.319267 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6695767e_4dfb_4603_8e8c_b410fe2c61c9.slice/crio-61cff45bc18848342d2e8941547f81ec9273d6b6a2b6215b152983330f862aff WatchSource:0}: Error finding container 61cff45bc18848342d2e8941547f81ec9273d6b6a2b6215b152983330f862aff: Status 404 returned error can't find the container with id 61cff45bc18848342d2e8941547f81ec9273d6b6a2b6215b152983330f862aff Sep 09 02:15:29.303619 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:29.303576 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b45446f48-4mb2l" event={"ID":"6695767e-4dfb-4603-8e8c-b410fe2c61c9","Type":"ContainerStarted","Data":"61cff45bc18848342d2e8941547f81ec9273d6b6a2b6215b152983330f862aff"} Sep 09 02:15:32.321138 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:32.321099 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b45446f48-4mb2l" event={"ID":"6695767e-4dfb-4603-8e8c-b410fe2c61c9","Type":"ContainerStarted","Data":"bc1cfd9ccd210dd88cb799925ecd6c177365ac1db20c7e8b5f411ba3f0ed7489"} Sep 09 02:15:33.326385 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:33.326354 2561 generic.go:356] "Generic (PLEG): container finished" podID="6695767e-4dfb-4603-8e8c-b410fe2c61c9" containerID="bc1cfd9ccd210dd88cb799925ecd6c177365ac1db20c7e8b5f411ba3f0ed7489" exitCode=0 Sep 09 02:15:33.326777 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:33.326396 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b45446f48-4mb2l" event={"ID":"6695767e-4dfb-4603-8e8c-b410fe2c61c9","Type":"ContainerDied","Data":"bc1cfd9ccd210dd88cb799925ecd6c177365ac1db20c7e8b5f411ba3f0ed7489"} Sep 09 02:15:34.331731 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:34.331697 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b45446f48-4mb2l" event={"ID":"6695767e-4dfb-4603-8e8c-b410fe2c61c9","Type":"ContainerStarted","Data":"7b1f0faab7dfd65ac734e90dca66b80e2f7887f089324a9be4cb06a33aa6d411"} Sep 09 02:15:34.332218 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:34.331802 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-b45446f48-4mb2l" Sep 09 02:15:34.357976 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:34.357920 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-b45446f48-4mb2l" podStartSLOduration=2.984426928 podStartE2EDuration="7.357903391s" totalImagesPullingTime="3.318193907s" totalInitContainerRuntime="1.055282556s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:15:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:15:34.356517438 +0000 UTC m=+293.731548847" watchObservedRunningTime="2026-09-09 02:15:34.357903391 +0000 UTC m=+293.732934799" Sep 09 02:15:41.003871 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:41.003837 2561 kubelet.go:1648] "Image garbage collection succeeded" Sep 09 02:15:45.337755 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:45.337726 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-b45446f48-4mb2l" Sep 09 02:15:47.902258 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:47.902219 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-67698fd555-d842h"] Sep 09 02:15:47.904313 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:47.904298 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-67698fd555-d842h" Sep 09 02:15:47.907361 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:47.906848 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 09 02:15:47.907361 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:47.906982 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 09 02:15:47.907361 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:47.907182 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Sep 09 02:15:47.907361 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:47.907263 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Sep 09 02:15:47.918738 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:47.918714 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-67698fd555-d842h"] Sep 09 02:15:48.041929 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:48.041893 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4glfn\" (UniqueName: \"kubernetes.io/projected/e9daf8a4-d9f9-4036-98eb-6d8d7feef5dc-kube-api-access-4glfn\") pod \"kube-auth-proxy-67698fd555-d842h\" (UID: \"e9daf8a4-d9f9-4036-98eb-6d8d7feef5dc\") " pod="openshift-ingress/kube-auth-proxy-67698fd555-d842h" Sep 09 02:15:48.042148 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:48.041965 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9daf8a4-d9f9-4036-98eb-6d8d7feef5dc-tls-certs\") pod \"kube-auth-proxy-67698fd555-d842h\" (UID: \"e9daf8a4-d9f9-4036-98eb-6d8d7feef5dc\") " pod="openshift-ingress/kube-auth-proxy-67698fd555-d842h" Sep 09 02:15:48.042148 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:48.042030 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e9daf8a4-d9f9-4036-98eb-6d8d7feef5dc-tmp\") pod \"kube-auth-proxy-67698fd555-d842h\" (UID: \"e9daf8a4-d9f9-4036-98eb-6d8d7feef5dc\") " pod="openshift-ingress/kube-auth-proxy-67698fd555-d842h" Sep 09 02:15:48.142979 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:48.142942 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4glfn\" (UniqueName: \"kubernetes.io/projected/e9daf8a4-d9f9-4036-98eb-6d8d7feef5dc-kube-api-access-4glfn\") pod \"kube-auth-proxy-67698fd555-d842h\" (UID: \"e9daf8a4-d9f9-4036-98eb-6d8d7feef5dc\") " pod="openshift-ingress/kube-auth-proxy-67698fd555-d842h" Sep 09 02:15:48.143148 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:48.142999 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9daf8a4-d9f9-4036-98eb-6d8d7feef5dc-tls-certs\") pod \"kube-auth-proxy-67698fd555-d842h\" (UID: \"e9daf8a4-d9f9-4036-98eb-6d8d7feef5dc\") " pod="openshift-ingress/kube-auth-proxy-67698fd555-d842h" Sep 09 02:15:48.143148 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:48.143050 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e9daf8a4-d9f9-4036-98eb-6d8d7feef5dc-tmp\") pod \"kube-auth-proxy-67698fd555-d842h\" (UID: \"e9daf8a4-d9f9-4036-98eb-6d8d7feef5dc\") " pod="openshift-ingress/kube-auth-proxy-67698fd555-d842h" Sep 09 02:15:48.145287 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:48.145262 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e9daf8a4-d9f9-4036-98eb-6d8d7feef5dc-tmp\") pod \"kube-auth-proxy-67698fd555-d842h\" (UID: \"e9daf8a4-d9f9-4036-98eb-6d8d7feef5dc\") " pod="openshift-ingress/kube-auth-proxy-67698fd555-d842h" Sep 09 02:15:48.145402 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:48.145384 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9daf8a4-d9f9-4036-98eb-6d8d7feef5dc-tls-certs\") pod \"kube-auth-proxy-67698fd555-d842h\" (UID: \"e9daf8a4-d9f9-4036-98eb-6d8d7feef5dc\") " pod="openshift-ingress/kube-auth-proxy-67698fd555-d842h" Sep 09 02:15:48.154903 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:48.154843 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4glfn\" (UniqueName: \"kubernetes.io/projected/e9daf8a4-d9f9-4036-98eb-6d8d7feef5dc-kube-api-access-4glfn\") pod \"kube-auth-proxy-67698fd555-d842h\" (UID: \"e9daf8a4-d9f9-4036-98eb-6d8d7feef5dc\") " pod="openshift-ingress/kube-auth-proxy-67698fd555-d842h" Sep 09 02:15:48.216302 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:48.216271 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-67698fd555-d842h" Sep 09 02:15:48.486028 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:48.485973 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-67698fd555-d842h"] Sep 09 02:15:48.488908 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:15:48.488877 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9daf8a4_d9f9_4036_98eb_6d8d7feef5dc.slice/crio-13c0dc75bd315462b32bd21cfad9e14539dbc795ac1bef413b925db5e5fce73f WatchSource:0}: Error finding container 13c0dc75bd315462b32bd21cfad9e14539dbc795ac1bef413b925db5e5fce73f: Status 404 returned error can't find the container with id 13c0dc75bd315462b32bd21cfad9e14539dbc795ac1bef413b925db5e5fce73f Sep 09 02:15:48.490563 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:48.490547 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 02:15:49.388906 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:49.388861 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-67698fd555-d842h" event={"ID":"e9daf8a4-d9f9-4036-98eb-6d8d7feef5dc","Type":"ContainerStarted","Data":"13c0dc75bd315462b32bd21cfad9e14539dbc795ac1bef413b925db5e5fce73f"} Sep 09 02:15:49.582365 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:49.582324 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc"] Sep 09 02:15:49.584767 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:49.584735 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc" Sep 09 02:15:49.588635 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:49.588608 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Sep 09 02:15:49.588775 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:49.588716 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 09 02:15:49.588985 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:49.588957 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Sep 09 02:15:49.589453 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:49.589297 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Sep 09 02:15:49.589453 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:49.589422 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-qg8zm\"" Sep 09 02:15:49.608049 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:49.607948 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc"] Sep 09 02:15:49.654336 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:49.654246 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fdk6\" (UniqueName: \"kubernetes.io/projected/9a998f24-c5f1-4147-bef2-38e023b536e6-kube-api-access-5fdk6\") pod \"istiod-openshift-gateway-7b7995d5cd-vd4cc\" (UID: \"9a998f24-c5f1-4147-bef2-38e023b536e6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc" Sep 09 02:15:49.654336 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:49.654289 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9a998f24-c5f1-4147-bef2-38e023b536e6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-vd4cc\" (UID: \"9a998f24-c5f1-4147-bef2-38e023b536e6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc" Sep 09 02:15:49.654336 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:49.654330 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9a998f24-c5f1-4147-bef2-38e023b536e6-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-vd4cc\" (UID: \"9a998f24-c5f1-4147-bef2-38e023b536e6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc" Sep 09 02:15:49.654568 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:49.654357 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9a998f24-c5f1-4147-bef2-38e023b536e6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-vd4cc\" (UID: \"9a998f24-c5f1-4147-bef2-38e023b536e6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc" Sep 09 02:15:49.654568 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:49.654461 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a998f24-c5f1-4147-bef2-38e023b536e6-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-vd4cc\" (UID: \"9a998f24-c5f1-4147-bef2-38e023b536e6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc" Sep 09 02:15:49.654568 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:49.654506 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a998f24-c5f1-4147-bef2-38e023b536e6-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-vd4cc\" (UID: \"9a998f24-c5f1-4147-bef2-38e023b536e6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc" Sep 09 02:15:49.654568 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:49.654525 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9a998f24-c5f1-4147-bef2-38e023b536e6-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-vd4cc\" (UID: \"9a998f24-c5f1-4147-bef2-38e023b536e6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc" Sep 09 02:15:49.756150 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:49.756111 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a998f24-c5f1-4147-bef2-38e023b536e6-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-vd4cc\" (UID: \"9a998f24-c5f1-4147-bef2-38e023b536e6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc" Sep 09 02:15:49.756346 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:49.756178 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a998f24-c5f1-4147-bef2-38e023b536e6-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-vd4cc\" (UID: \"9a998f24-c5f1-4147-bef2-38e023b536e6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc" Sep 09 02:15:49.756346 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:49.756206 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9a998f24-c5f1-4147-bef2-38e023b536e6-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-vd4cc\" (UID: \"9a998f24-c5f1-4147-bef2-38e023b536e6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc" Sep 09 02:15:49.756346 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:49.756235 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fdk6\" (UniqueName: \"kubernetes.io/projected/9a998f24-c5f1-4147-bef2-38e023b536e6-kube-api-access-5fdk6\") pod \"istiod-openshift-gateway-7b7995d5cd-vd4cc\" (UID: \"9a998f24-c5f1-4147-bef2-38e023b536e6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc" Sep 09 02:15:49.756346 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:49.756258 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9a998f24-c5f1-4147-bef2-38e023b536e6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-vd4cc\" (UID: \"9a998f24-c5f1-4147-bef2-38e023b536e6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc" Sep 09 02:15:49.756346 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:49.756298 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9a998f24-c5f1-4147-bef2-38e023b536e6-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-vd4cc\" (UID: \"9a998f24-c5f1-4147-bef2-38e023b536e6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc" Sep 09 02:15:49.756346 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:49.756320 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9a998f24-c5f1-4147-bef2-38e023b536e6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-vd4cc\" (UID: \"9a998f24-c5f1-4147-bef2-38e023b536e6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc" Sep 09 02:15:49.757402 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:49.757345 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9a998f24-c5f1-4147-bef2-38e023b536e6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-vd4cc\" (UID: \"9a998f24-c5f1-4147-bef2-38e023b536e6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc" Sep 09 02:15:49.759507 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:49.759468 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9a998f24-c5f1-4147-bef2-38e023b536e6-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-vd4cc\" (UID: \"9a998f24-c5f1-4147-bef2-38e023b536e6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc" Sep 09 02:15:49.759598 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:49.759585 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a998f24-c5f1-4147-bef2-38e023b536e6-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-vd4cc\" (UID: \"9a998f24-c5f1-4147-bef2-38e023b536e6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc" Sep 09 02:15:49.759994 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:49.759974 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9a998f24-c5f1-4147-bef2-38e023b536e6-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-vd4cc\" (UID: \"9a998f24-c5f1-4147-bef2-38e023b536e6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc" Sep 09 02:15:49.760080 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:49.760054 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9a998f24-c5f1-4147-bef2-38e023b536e6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-vd4cc\" (UID: \"9a998f24-c5f1-4147-bef2-38e023b536e6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc" Sep 09 02:15:49.764907 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:49.764876 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fdk6\" (UniqueName: \"kubernetes.io/projected/9a998f24-c5f1-4147-bef2-38e023b536e6-kube-api-access-5fdk6\") pod \"istiod-openshift-gateway-7b7995d5cd-vd4cc\" (UID: \"9a998f24-c5f1-4147-bef2-38e023b536e6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc" Sep 09 02:15:49.765058 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:49.765040 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a998f24-c5f1-4147-bef2-38e023b536e6-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-vd4cc\" (UID: \"9a998f24-c5f1-4147-bef2-38e023b536e6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc" Sep 09 02:15:49.898860 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:49.898827 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc" Sep 09 02:15:50.181458 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:50.181396 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc"] Sep 09 02:15:50.187272 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:15:50.187231 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a998f24_c5f1_4147_bef2_38e023b536e6.slice/crio-be1b8353e0bc25369f372f8251c31df23d4c9b9f922933dc6994762a3615f1b6 WatchSource:0}: Error finding container be1b8353e0bc25369f372f8251c31df23d4c9b9f922933dc6994762a3615f1b6: Status 404 returned error can't find the container with id be1b8353e0bc25369f372f8251c31df23d4c9b9f922933dc6994762a3615f1b6 Sep 09 02:15:50.394612 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:50.394576 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc" event={"ID":"9a998f24-c5f1-4147-bef2-38e023b536e6","Type":"ContainerStarted","Data":"be1b8353e0bc25369f372f8251c31df23d4c9b9f922933dc6994762a3615f1b6"} Sep 09 02:15:52.410869 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:52.410827 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-67698fd555-d842h" event={"ID":"e9daf8a4-d9f9-4036-98eb-6d8d7feef5dc","Type":"ContainerStarted","Data":"c97d1fe392b5752d3294ef0e3ee26fe7c0b0a7cedd30a4233b71d0c153947087"} Sep 09 02:15:52.429279 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:52.429207 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-67698fd555-d842h" podStartSLOduration=2.151611375 podStartE2EDuration="5.429186019s" totalImagesPullingTime="3.277574644s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:15:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:15:52.42855649 +0000 UTC m=+311.803587911" watchObservedRunningTime="2026-09-09 02:15:52.429186019 +0000 UTC m=+311.804217430" Sep 09 02:15:53.786803 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:53.786767 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 09 02:15:53.787066 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:53.786836 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 09 02:15:54.419801 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:54.419768 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc" event={"ID":"9a998f24-c5f1-4147-bef2-38e023b536e6","Type":"ContainerStarted","Data":"fc6814593b5d9e5b98404efb43ceb691b7f63ac5a7d269eea128c867f06bd28b"} Sep 09 02:15:54.420163 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:54.419950 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc" Sep 09 02:15:54.421782 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:54.421750 2561 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-vd4cc container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 09 02:15:54.421892 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:54.421804 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc" podUID="9a998f24-c5f1-4147-bef2-38e023b536e6" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 02:15:54.439880 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:54.439825 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc" podStartSLOduration=1.843094037 podStartE2EDuration="5.439809344s" totalImagesPullingTime="3.596715307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:15:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:15:54.43852952 +0000 UTC m=+313.813560929" watchObservedRunningTime="2026-09-09 02:15:54.439809344 +0000 UTC m=+313.814840753" Sep 09 02:15:55.424484 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:55.424446 2561 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-vd4cc container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 09 02:15:55.424877 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:55.424510 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc" podUID="9a998f24-c5f1-4147-bef2-38e023b536e6" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 02:15:58.424516 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:15:58.424484 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc" Sep 09 02:16:07.959692 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:07.959651 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-kzz89"] Sep 09 02:16:07.963521 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:07.963497 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzz89" Sep 09 02:16:07.966406 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:07.966379 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-nmjbp\"" Sep 09 02:16:07.966525 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:07.966498 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Sep 09 02:16:07.975305 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:07.975274 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-kzz89"] Sep 09 02:16:07.977629 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:07.977600 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-t4cmm"] Sep 09 02:16:07.981137 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:07.981114 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-t4cmm" Sep 09 02:16:07.983396 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:07.983343 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Sep 09 02:16:07.983513 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:07.983440 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-h26r2\"" Sep 09 02:16:07.993950 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:07.993924 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-t4cmm"] Sep 09 02:16:08.117470 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:08.117434 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b8176964-35e0-457b-b4d0-2719b5ebda10-cert\") pod \"kserve-controller-manager-d6f4b65b8-kzz89\" (UID: \"b8176964-35e0-457b-b4d0-2719b5ebda10\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzz89" Sep 09 02:16:08.117660 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:08.117488 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e8205051-6be7-45cc-9e3c-0182117ead59-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-t4cmm\" (UID: \"e8205051-6be7-45cc-9e3c-0182117ead59\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-t4cmm" Sep 09 02:16:08.117660 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:08.117571 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xthq\" (UniqueName: \"kubernetes.io/projected/e8205051-6be7-45cc-9e3c-0182117ead59-kube-api-access-7xthq\") pod \"llmisvc-controller-manager-777c6bd7d5-t4cmm\" (UID: \"e8205051-6be7-45cc-9e3c-0182117ead59\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-t4cmm" Sep 09 02:16:08.117660 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:08.117606 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wr227\" (UniqueName: \"kubernetes.io/projected/b8176964-35e0-457b-b4d0-2719b5ebda10-kube-api-access-wr227\") pod \"kserve-controller-manager-d6f4b65b8-kzz89\" (UID: \"b8176964-35e0-457b-b4d0-2719b5ebda10\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzz89" Sep 09 02:16:08.218261 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:08.218182 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b8176964-35e0-457b-b4d0-2719b5ebda10-cert\") pod \"kserve-controller-manager-d6f4b65b8-kzz89\" (UID: \"b8176964-35e0-457b-b4d0-2719b5ebda10\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzz89" Sep 09 02:16:08.218261 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:08.218235 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e8205051-6be7-45cc-9e3c-0182117ead59-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-t4cmm\" (UID: \"e8205051-6be7-45cc-9e3c-0182117ead59\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-t4cmm" Sep 09 02:16:08.218486 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:08.218268 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xthq\" (UniqueName: \"kubernetes.io/projected/e8205051-6be7-45cc-9e3c-0182117ead59-kube-api-access-7xthq\") pod \"llmisvc-controller-manager-777c6bd7d5-t4cmm\" (UID: \"e8205051-6be7-45cc-9e3c-0182117ead59\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-t4cmm" Sep 09 02:16:08.218486 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:08.218297 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wr227\" (UniqueName: \"kubernetes.io/projected/b8176964-35e0-457b-b4d0-2719b5ebda10-kube-api-access-wr227\") pod \"kserve-controller-manager-d6f4b65b8-kzz89\" (UID: \"b8176964-35e0-457b-b4d0-2719b5ebda10\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzz89" Sep 09 02:16:08.220663 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:08.220629 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b8176964-35e0-457b-b4d0-2719b5ebda10-cert\") pod \"kserve-controller-manager-d6f4b65b8-kzz89\" (UID: \"b8176964-35e0-457b-b4d0-2719b5ebda10\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzz89" Sep 09 02:16:08.220663 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:08.220656 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e8205051-6be7-45cc-9e3c-0182117ead59-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-t4cmm\" (UID: \"e8205051-6be7-45cc-9e3c-0182117ead59\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-t4cmm" Sep 09 02:16:08.233461 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:08.233432 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xthq\" (UniqueName: \"kubernetes.io/projected/e8205051-6be7-45cc-9e3c-0182117ead59-kube-api-access-7xthq\") pod \"llmisvc-controller-manager-777c6bd7d5-t4cmm\" (UID: \"e8205051-6be7-45cc-9e3c-0182117ead59\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-t4cmm" Sep 09 02:16:08.234382 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:08.234363 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wr227\" (UniqueName: \"kubernetes.io/projected/b8176964-35e0-457b-b4d0-2719b5ebda10-kube-api-access-wr227\") pod \"kserve-controller-manager-d6f4b65b8-kzz89\" (UID: \"b8176964-35e0-457b-b4d0-2719b5ebda10\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzz89" Sep 09 02:16:08.274689 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:08.274663 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzz89" Sep 09 02:16:08.293584 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:08.293555 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-t4cmm" Sep 09 02:16:08.558545 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:08.558390 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-kzz89"] Sep 09 02:16:08.561239 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:16:08.561212 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8176964_35e0_457b_b4d0_2719b5ebda10.slice/crio-ef58e20e41df5e58a00196d03f4110d7b92a26361d73fa21eb417fc2da233875 WatchSource:0}: Error finding container ef58e20e41df5e58a00196d03f4110d7b92a26361d73fa21eb417fc2da233875: Status 404 returned error can't find the container with id ef58e20e41df5e58a00196d03f4110d7b92a26361d73fa21eb417fc2da233875 Sep 09 02:16:08.579653 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:08.579631 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-t4cmm"] Sep 09 02:16:08.581604 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:16:08.581578 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode8205051_6be7_45cc_9e3c_0182117ead59.slice/crio-6f0a0fe91ff8a228dab021aa3fd796b9bddfc1c20a67bf94fdc04b655a5fc239 WatchSource:0}: Error finding container 6f0a0fe91ff8a228dab021aa3fd796b9bddfc1c20a67bf94fdc04b655a5fc239: Status 404 returned error can't find the container with id 6f0a0fe91ff8a228dab021aa3fd796b9bddfc1c20a67bf94fdc04b655a5fc239 Sep 09 02:16:09.478429 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:09.478385 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-t4cmm" event={"ID":"e8205051-6be7-45cc-9e3c-0182117ead59","Type":"ContainerStarted","Data":"6f0a0fe91ff8a228dab021aa3fd796b9bddfc1c20a67bf94fdc04b655a5fc239"} Sep 09 02:16:09.479946 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:09.479904 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzz89" event={"ID":"b8176964-35e0-457b-b4d0-2719b5ebda10","Type":"ContainerStarted","Data":"ef58e20e41df5e58a00196d03f4110d7b92a26361d73fa21eb417fc2da233875"} Sep 09 02:16:09.679832 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:09.679794 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc"] Sep 09 02:16:09.680087 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:09.680061 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc" podUID="9a998f24-c5f1-4147-bef2-38e023b536e6" containerName="discovery" containerID="cri-o://fc6814593b5d9e5b98404efb43ceb691b7f63ac5a7d269eea128c867f06bd28b" gracePeriod=30 Sep 09 02:16:10.053174 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:10.053061 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc" Sep 09 02:16:10.234786 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:10.234750 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9a998f24-c5f1-4147-bef2-38e023b536e6-cacerts\") pod \"9a998f24-c5f1-4147-bef2-38e023b536e6\" (UID: \"9a998f24-c5f1-4147-bef2-38e023b536e6\") " Sep 09 02:16:10.234786 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:10.234795 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9a998f24-c5f1-4147-bef2-38e023b536e6-istio-csr-dns-cert\") pod \"9a998f24-c5f1-4147-bef2-38e023b536e6\" (UID: \"9a998f24-c5f1-4147-bef2-38e023b536e6\") " Sep 09 02:16:10.235037 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:10.234827 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9a998f24-c5f1-4147-bef2-38e023b536e6-istio-csr-ca-configmap\") pod \"9a998f24-c5f1-4147-bef2-38e023b536e6\" (UID: \"9a998f24-c5f1-4147-bef2-38e023b536e6\") " Sep 09 02:16:10.235037 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:10.234938 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a998f24-c5f1-4147-bef2-38e023b536e6-local-certs\") pod \"9a998f24-c5f1-4147-bef2-38e023b536e6\" (UID: \"9a998f24-c5f1-4147-bef2-38e023b536e6\") " Sep 09 02:16:10.235152 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:10.235066 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fdk6\" (UniqueName: \"kubernetes.io/projected/9a998f24-c5f1-4147-bef2-38e023b536e6-kube-api-access-5fdk6\") pod \"9a998f24-c5f1-4147-bef2-38e023b536e6\" (UID: \"9a998f24-c5f1-4147-bef2-38e023b536e6\") " Sep 09 02:16:10.235241 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:10.235216 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9a998f24-c5f1-4147-bef2-38e023b536e6-istio-kubeconfig\") pod \"9a998f24-c5f1-4147-bef2-38e023b536e6\" (UID: \"9a998f24-c5f1-4147-bef2-38e023b536e6\") " Sep 09 02:16:10.235372 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:10.235256 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a998f24-c5f1-4147-bef2-38e023b536e6-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "9a998f24-c5f1-4147-bef2-38e023b536e6" (UID: "9a998f24-c5f1-4147-bef2-38e023b536e6"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 02:16:10.235372 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:10.235269 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a998f24-c5f1-4147-bef2-38e023b536e6-istio-token\") pod \"9a998f24-c5f1-4147-bef2-38e023b536e6\" (UID: \"9a998f24-c5f1-4147-bef2-38e023b536e6\") " Sep 09 02:16:10.235581 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:10.235552 2561 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9a998f24-c5f1-4147-bef2-38e023b536e6-istio-csr-ca-configmap\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:16:10.238117 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:10.238081 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a998f24-c5f1-4147-bef2-38e023b536e6-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "9a998f24-c5f1-4147-bef2-38e023b536e6" (UID: "9a998f24-c5f1-4147-bef2-38e023b536e6"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 02:16:10.238537 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:10.238496 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a998f24-c5f1-4147-bef2-38e023b536e6-local-certs" (OuterVolumeSpecName: "local-certs") pod "9a998f24-c5f1-4147-bef2-38e023b536e6" (UID: "9a998f24-c5f1-4147-bef2-38e023b536e6"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 02:16:10.238654 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:10.238610 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a998f24-c5f1-4147-bef2-38e023b536e6-istio-token" (OuterVolumeSpecName: "istio-token") pod "9a998f24-c5f1-4147-bef2-38e023b536e6" (UID: "9a998f24-c5f1-4147-bef2-38e023b536e6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 02:16:10.239364 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:10.239035 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a998f24-c5f1-4147-bef2-38e023b536e6-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "9a998f24-c5f1-4147-bef2-38e023b536e6" (UID: "9a998f24-c5f1-4147-bef2-38e023b536e6"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 02:16:10.239364 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:10.239103 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a998f24-c5f1-4147-bef2-38e023b536e6-cacerts" (OuterVolumeSpecName: "cacerts") pod "9a998f24-c5f1-4147-bef2-38e023b536e6" (UID: "9a998f24-c5f1-4147-bef2-38e023b536e6"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 02:16:10.239364 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:10.239223 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a998f24-c5f1-4147-bef2-38e023b536e6-kube-api-access-5fdk6" (OuterVolumeSpecName: "kube-api-access-5fdk6") pod "9a998f24-c5f1-4147-bef2-38e023b536e6" (UID: "9a998f24-c5f1-4147-bef2-38e023b536e6"). InnerVolumeSpecName "kube-api-access-5fdk6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 02:16:10.336386 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:10.336343 2561 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9a998f24-c5f1-4147-bef2-38e023b536e6-istio-kubeconfig\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:16:10.336386 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:10.336380 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a998f24-c5f1-4147-bef2-38e023b536e6-istio-token\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:16:10.336386 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:10.336391 2561 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9a998f24-c5f1-4147-bef2-38e023b536e6-cacerts\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:16:10.336667 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:10.336401 2561 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9a998f24-c5f1-4147-bef2-38e023b536e6-istio-csr-dns-cert\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:16:10.336667 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:10.336416 2561 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a998f24-c5f1-4147-bef2-38e023b536e6-local-certs\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:16:10.336667 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:10.336430 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fdk6\" (UniqueName: \"kubernetes.io/projected/9a998f24-c5f1-4147-bef2-38e023b536e6-kube-api-access-5fdk6\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:16:10.485987 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:10.485900 2561 generic.go:356] "Generic (PLEG): container finished" podID="9a998f24-c5f1-4147-bef2-38e023b536e6" containerID="fc6814593b5d9e5b98404efb43ceb691b7f63ac5a7d269eea128c867f06bd28b" exitCode=0 Sep 09 02:16:10.485987 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:10.485960 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc" Sep 09 02:16:10.486551 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:10.485992 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc" event={"ID":"9a998f24-c5f1-4147-bef2-38e023b536e6","Type":"ContainerDied","Data":"fc6814593b5d9e5b98404efb43ceb691b7f63ac5a7d269eea128c867f06bd28b"} Sep 09 02:16:10.486551 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:10.486060 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc" event={"ID":"9a998f24-c5f1-4147-bef2-38e023b536e6","Type":"ContainerDied","Data":"be1b8353e0bc25369f372f8251c31df23d4c9b9f922933dc6994762a3615f1b6"} Sep 09 02:16:10.486551 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:10.486086 2561 scope.go:117] "RemoveContainer" containerID="fc6814593b5d9e5b98404efb43ceb691b7f63ac5a7d269eea128c867f06bd28b" Sep 09 02:16:10.530612 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:10.530572 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc"] Sep 09 02:16:10.536852 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:10.536817 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vd4cc"] Sep 09 02:16:10.801965 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:10.801822 2561 scope.go:117] "RemoveContainer" containerID="fc6814593b5d9e5b98404efb43ceb691b7f63ac5a7d269eea128c867f06bd28b" Sep 09 02:16:10.802245 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:16:10.802222 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc6814593b5d9e5b98404efb43ceb691b7f63ac5a7d269eea128c867f06bd28b\": container with ID starting with fc6814593b5d9e5b98404efb43ceb691b7f63ac5a7d269eea128c867f06bd28b not found: ID does not exist" containerID="fc6814593b5d9e5b98404efb43ceb691b7f63ac5a7d269eea128c867f06bd28b" Sep 09 02:16:10.802304 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:10.802256 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc6814593b5d9e5b98404efb43ceb691b7f63ac5a7d269eea128c867f06bd28b"} err="failed to get container status \"fc6814593b5d9e5b98404efb43ceb691b7f63ac5a7d269eea128c867f06bd28b\": rpc error: code = NotFound desc = could not find container \"fc6814593b5d9e5b98404efb43ceb691b7f63ac5a7d269eea128c867f06bd28b\": container with ID starting with fc6814593b5d9e5b98404efb43ceb691b7f63ac5a7d269eea128c867f06bd28b not found: ID does not exist" Sep 09 02:16:11.111945 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:11.111862 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a998f24-c5f1-4147-bef2-38e023b536e6" path="/var/lib/kubelet/pods/9a998f24-c5f1-4147-bef2-38e023b536e6/volumes" Sep 09 02:16:11.494132 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:11.494095 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzz89" event={"ID":"b8176964-35e0-457b-b4d0-2719b5ebda10","Type":"ContainerStarted","Data":"d825a4b69fcdbad3c5073e3cb605c75f0458aada6ef2e8e37cb2314e897daf9e"} Sep 09 02:16:11.494615 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:11.494248 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzz89" Sep 09 02:16:11.529680 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:11.529625 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzz89" podStartSLOduration=2.241290896 podStartE2EDuration="4.529605068s" totalImagesPullingTime="2.288314172s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:16:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:16:11.527569369 +0000 UTC m=+330.902600791" watchObservedRunningTime="2026-09-09 02:16:11.529605068 +0000 UTC m=+330.904636476" Sep 09 02:16:12.499378 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:12.499343 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-t4cmm" event={"ID":"e8205051-6be7-45cc-9e3c-0182117ead59","Type":"ContainerStarted","Data":"50d9101d017d2ebbb51a0b3005bdacf1e41582b98bd25dbb538887407832c7a3"} Sep 09 02:16:12.499760 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:12.499512 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-t4cmm" Sep 09 02:16:12.517870 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:12.517814 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-t4cmm" podStartSLOduration=2.50234618 podStartE2EDuration="5.517796049s" totalImagesPullingTime="3.015449869s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:16:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:16:12.516770165 +0000 UTC m=+331.891801575" watchObservedRunningTime="2026-09-09 02:16:12.517796049 +0000 UTC m=+331.892827458" Sep 09 02:16:42.504370 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:42.504335 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzz89" Sep 09 02:16:43.505933 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:16:43.505897 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-t4cmm" Sep 09 02:17:14.956978 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:14.956941 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-dljqg"] Sep 09 02:17:14.957445 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:14.957263 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a998f24-c5f1-4147-bef2-38e023b536e6" containerName="discovery" Sep 09 02:17:14.957445 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:14.957278 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a998f24-c5f1-4147-bef2-38e023b536e6" containerName="discovery" Sep 09 02:17:14.957445 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:14.957333 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a998f24-c5f1-4147-bef2-38e023b536e6" containerName="discovery" Sep 09 02:17:14.959783 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:14.959766 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dljqg" Sep 09 02:17:14.965381 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:14.965359 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 09 02:17:14.965831 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:14.965807 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-qfbm8\"" Sep 09 02:17:14.965968 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:14.965873 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 09 02:17:14.969289 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:14.969267 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-dljqg"] Sep 09 02:17:15.066271 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:15.066233 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5pzx\" (UniqueName: \"kubernetes.io/projected/bbf56ca1-18be-45d2-b5e7-4d4a3473b2bb-kube-api-access-z5pzx\") pod \"limitador-operator-controller-manager-74495c69f8-dljqg\" (UID: \"bbf56ca1-18be-45d2-b5e7-4d4a3473b2bb\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dljqg" Sep 09 02:17:15.167472 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:15.167429 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5pzx\" (UniqueName: \"kubernetes.io/projected/bbf56ca1-18be-45d2-b5e7-4d4a3473b2bb-kube-api-access-z5pzx\") pod \"limitador-operator-controller-manager-74495c69f8-dljqg\" (UID: \"bbf56ca1-18be-45d2-b5e7-4d4a3473b2bb\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dljqg" Sep 09 02:17:15.179504 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:15.179472 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5pzx\" (UniqueName: \"kubernetes.io/projected/bbf56ca1-18be-45d2-b5e7-4d4a3473b2bb-kube-api-access-z5pzx\") pod \"limitador-operator-controller-manager-74495c69f8-dljqg\" (UID: \"bbf56ca1-18be-45d2-b5e7-4d4a3473b2bb\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dljqg" Sep 09 02:17:15.275654 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:15.275563 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dljqg" Sep 09 02:17:15.545935 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:15.545891 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-dljqg"] Sep 09 02:17:15.548309 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:17:15.548278 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbf56ca1_18be_45d2_b5e7_4d4a3473b2bb.slice/crio-24637adb6445b8d85827ccfa4eda85a00254c98890f0036f1cdf87ce9e6112a9 WatchSource:0}: Error finding container 24637adb6445b8d85827ccfa4eda85a00254c98890f0036f1cdf87ce9e6112a9: Status 404 returned error can't find the container with id 24637adb6445b8d85827ccfa4eda85a00254c98890f0036f1cdf87ce9e6112a9 Sep 09 02:17:15.732805 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:15.732766 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dljqg" event={"ID":"bbf56ca1-18be-45d2-b5e7-4d4a3473b2bb","Type":"ContainerStarted","Data":"24637adb6445b8d85827ccfa4eda85a00254c98890f0036f1cdf87ce9e6112a9"} Sep 09 02:17:17.228839 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:17.228809 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5hm9l"] Sep 09 02:17:17.231409 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:17.231387 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5hm9l" Sep 09 02:17:17.235089 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:17.235063 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-bwqt5\"" Sep 09 02:17:17.235202 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:17.235177 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Sep 09 02:17:17.260566 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:17.260535 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5hm9l"] Sep 09 02:17:17.386184 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:17.386141 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmqw8\" (UniqueName: \"kubernetes.io/projected/e99b236b-e562-47ad-b8e9-892f95166806-kube-api-access-jmqw8\") pod \"dns-operator-controller-manager-7b84cd7c9c-5hm9l\" (UID: \"e99b236b-e562-47ad-b8e9-892f95166806\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5hm9l" Sep 09 02:17:17.486877 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:17.486797 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmqw8\" (UniqueName: \"kubernetes.io/projected/e99b236b-e562-47ad-b8e9-892f95166806-kube-api-access-jmqw8\") pod \"dns-operator-controller-manager-7b84cd7c9c-5hm9l\" (UID: \"e99b236b-e562-47ad-b8e9-892f95166806\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5hm9l" Sep 09 02:17:17.521192 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:17.521154 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmqw8\" (UniqueName: \"kubernetes.io/projected/e99b236b-e562-47ad-b8e9-892f95166806-kube-api-access-jmqw8\") pod \"dns-operator-controller-manager-7b84cd7c9c-5hm9l\" (UID: \"e99b236b-e562-47ad-b8e9-892f95166806\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5hm9l" Sep 09 02:17:17.542945 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:17.542916 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5hm9l" Sep 09 02:17:18.219397 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:18.219365 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5hm9l"] Sep 09 02:17:18.416566 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:17:18.416530 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode99b236b_e562_47ad_b8e9_892f95166806.slice/crio-a9c83ddbcb346e8721901cd8b58eab1c90986db5a8c34ea5772209418a593ed6 WatchSource:0}: Error finding container a9c83ddbcb346e8721901cd8b58eab1c90986db5a8c34ea5772209418a593ed6: Status 404 returned error can't find the container with id a9c83ddbcb346e8721901cd8b58eab1c90986db5a8c34ea5772209418a593ed6 Sep 09 02:17:18.750603 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:18.750503 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5hm9l" event={"ID":"e99b236b-e562-47ad-b8e9-892f95166806","Type":"ContainerStarted","Data":"a9c83ddbcb346e8721901cd8b58eab1c90986db5a8c34ea5772209418a593ed6"} Sep 09 02:17:18.752054 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:18.752003 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dljqg" event={"ID":"bbf56ca1-18be-45d2-b5e7-4d4a3473b2bb","Type":"ContainerStarted","Data":"c4b92ad043630b867b59a8470aecfa8531a760447cb511bb7d58f2a5da21b3de"} Sep 09 02:17:18.752199 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:18.752150 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dljqg" Sep 09 02:17:18.774772 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:18.774698 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dljqg" podStartSLOduration=1.860464127 podStartE2EDuration="4.774682013s" totalImagesPullingTime="2.914217886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:17:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:17:18.774576869 +0000 UTC m=+398.149608279" watchObservedRunningTime="2026-09-09 02:17:18.774682013 +0000 UTC m=+398.149713422" Sep 09 02:17:20.764105 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:20.764064 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5hm9l" event={"ID":"e99b236b-e562-47ad-b8e9-892f95166806","Type":"ContainerStarted","Data":"533ba5fa6f1d3b8dff272226ca5dda4dfb99f0ac63958685c186092bcf7a771a"} Sep 09 02:17:20.764485 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:20.764187 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5hm9l" Sep 09 02:17:20.783430 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:20.783378 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5hm9l" podStartSLOduration=1.866680847 podStartE2EDuration="3.783363396s" totalImagesPullingTime="1.916682549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:17:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:17:20.781523973 +0000 UTC m=+400.156555382" watchObservedRunningTime="2026-09-09 02:17:20.783363396 +0000 UTC m=+400.158394804" Sep 09 02:17:22.005865 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:22.005833 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s88fz"] Sep 09 02:17:22.013263 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:22.013235 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s88fz" Sep 09 02:17:22.015760 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:22.015734 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-qp656\"" Sep 09 02:17:22.025995 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:22.025973 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfhwc\" (UniqueName: \"kubernetes.io/projected/235fb8c7-2620-4830-b04d-cee7203c8232-kube-api-access-sfhwc\") pod \"kuadrant-operator-controller-manager-76f8f686bf-s88fz\" (UID: \"235fb8c7-2620-4830-b04d-cee7203c8232\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s88fz" Sep 09 02:17:22.026106 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:22.026030 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/235fb8c7-2620-4830-b04d-cee7203c8232-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-s88fz\" (UID: \"235fb8c7-2620-4830-b04d-cee7203c8232\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s88fz" Sep 09 02:17:22.031401 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:22.031377 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s88fz"] Sep 09 02:17:22.126422 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:22.126392 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfhwc\" (UniqueName: \"kubernetes.io/projected/235fb8c7-2620-4830-b04d-cee7203c8232-kube-api-access-sfhwc\") pod \"kuadrant-operator-controller-manager-76f8f686bf-s88fz\" (UID: \"235fb8c7-2620-4830-b04d-cee7203c8232\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s88fz" Sep 09 02:17:22.126612 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:22.126434 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/235fb8c7-2620-4830-b04d-cee7203c8232-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-s88fz\" (UID: \"235fb8c7-2620-4830-b04d-cee7203c8232\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s88fz" Sep 09 02:17:22.126788 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:22.126761 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/235fb8c7-2620-4830-b04d-cee7203c8232-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-s88fz\" (UID: \"235fb8c7-2620-4830-b04d-cee7203c8232\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s88fz" Sep 09 02:17:22.135911 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:22.135894 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfhwc\" (UniqueName: \"kubernetes.io/projected/235fb8c7-2620-4830-b04d-cee7203c8232-kube-api-access-sfhwc\") pod \"kuadrant-operator-controller-manager-76f8f686bf-s88fz\" (UID: \"235fb8c7-2620-4830-b04d-cee7203c8232\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s88fz" Sep 09 02:17:22.323358 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:22.323267 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s88fz" Sep 09 02:17:22.596998 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:22.596912 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s88fz"] Sep 09 02:17:22.601763 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:17:22.601722 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod235fb8c7_2620_4830_b04d_cee7203c8232.slice/crio-7f42f22e02850aecfdb635e998788b1af12432929ce5921ede21a5a2e9655b0d WatchSource:0}: Error finding container 7f42f22e02850aecfdb635e998788b1af12432929ce5921ede21a5a2e9655b0d: Status 404 returned error can't find the container with id 7f42f22e02850aecfdb635e998788b1af12432929ce5921ede21a5a2e9655b0d Sep 09 02:17:22.774815 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:22.774777 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s88fz" event={"ID":"235fb8c7-2620-4830-b04d-cee7203c8232","Type":"ContainerStarted","Data":"7f42f22e02850aecfdb635e998788b1af12432929ce5921ede21a5a2e9655b0d"} Sep 09 02:17:25.174597 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:25.174561 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-plmhn"] Sep 09 02:17:25.176911 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:25.176885 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-plmhn" Sep 09 02:17:25.179550 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:25.179527 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-4h8kb\"" Sep 09 02:17:25.191219 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:25.191199 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-plmhn"] Sep 09 02:17:25.245300 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:25.245259 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4ph6\" (UniqueName: \"kubernetes.io/projected/78224fa0-c4b4-4bc5-8208-359681e17c53-kube-api-access-r4ph6\") pod \"authorino-operator-76b478bd78-plmhn\" (UID: \"78224fa0-c4b4-4bc5-8208-359681e17c53\") " pod="kuadrant-system/authorino-operator-76b478bd78-plmhn" Sep 09 02:17:25.346280 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:25.346244 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4ph6\" (UniqueName: \"kubernetes.io/projected/78224fa0-c4b4-4bc5-8208-359681e17c53-kube-api-access-r4ph6\") pod \"authorino-operator-76b478bd78-plmhn\" (UID: \"78224fa0-c4b4-4bc5-8208-359681e17c53\") " pod="kuadrant-system/authorino-operator-76b478bd78-plmhn" Sep 09 02:17:25.356110 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:25.356069 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4ph6\" (UniqueName: \"kubernetes.io/projected/78224fa0-c4b4-4bc5-8208-359681e17c53-kube-api-access-r4ph6\") pod \"authorino-operator-76b478bd78-plmhn\" (UID: \"78224fa0-c4b4-4bc5-8208-359681e17c53\") " pod="kuadrant-system/authorino-operator-76b478bd78-plmhn" Sep 09 02:17:25.490504 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:25.490425 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-plmhn" Sep 09 02:17:25.778234 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:25.778074 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-plmhn"] Sep 09 02:17:26.925478 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:17:26.925434 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78224fa0_c4b4_4bc5_8208_359681e17c53.slice/crio-fa6cd2b545f757d2515478c383b89d9f829803c9f9e413da39bc4131adb38e6b WatchSource:0}: Error finding container fa6cd2b545f757d2515478c383b89d9f829803c9f9e413da39bc4131adb38e6b: Status 404 returned error can't find the container with id fa6cd2b545f757d2515478c383b89d9f829803c9f9e413da39bc4131adb38e6b Sep 09 02:17:27.798638 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:27.798602 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-plmhn" event={"ID":"78224fa0-c4b4-4bc5-8208-359681e17c53","Type":"ContainerStarted","Data":"fa6cd2b545f757d2515478c383b89d9f829803c9f9e413da39bc4131adb38e6b"} Sep 09 02:17:27.800245 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:27.800218 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s88fz" event={"ID":"235fb8c7-2620-4830-b04d-cee7203c8232","Type":"ContainerStarted","Data":"9b4a89b5a340d58e5d13d4bc8a7de3d2ab9951dde1519bd193a2fc9319eccfbc"} Sep 09 02:17:27.800359 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:27.800288 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s88fz" Sep 09 02:17:27.821121 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:27.821075 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s88fz" podStartSLOduration=2.457345319 podStartE2EDuration="6.821059511s" totalImagesPullingTime="4.363714192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:17:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:17:27.819173724 +0000 UTC m=+407.194205132" watchObservedRunningTime="2026-09-09 02:17:27.821059511 +0000 UTC m=+407.196090919" Sep 09 02:17:29.701621 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:29.701582 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-mfs22"] Sep 09 02:17:29.703944 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:29.703928 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-mfs22" Sep 09 02:17:29.706608 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:29.706561 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Sep 09 02:17:29.706748 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:29.706605 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Sep 09 02:17:29.706748 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:29.706662 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-dz7zn\"" Sep 09 02:17:29.716777 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:29.716749 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-mfs22"] Sep 09 02:17:29.760377 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:29.760346 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dljqg" Sep 09 02:17:29.777458 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:29.777422 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8ea7a2f1-03b4-4665-8286-5ef11d4eb9df-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-mfs22\" (UID: \"8ea7a2f1-03b4-4665-8286-5ef11d4eb9df\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-mfs22" Sep 09 02:17:29.777640 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:29.777591 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8gnl\" (UniqueName: \"kubernetes.io/projected/8ea7a2f1-03b4-4665-8286-5ef11d4eb9df-kube-api-access-d8gnl\") pod \"kuadrant-console-plugin-fbdd4f445-mfs22\" (UID: \"8ea7a2f1-03b4-4665-8286-5ef11d4eb9df\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-mfs22" Sep 09 02:17:29.777724 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:29.777707 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ea7a2f1-03b4-4665-8286-5ef11d4eb9df-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-mfs22\" (UID: \"8ea7a2f1-03b4-4665-8286-5ef11d4eb9df\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-mfs22" Sep 09 02:17:29.810599 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:29.810561 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-plmhn" event={"ID":"78224fa0-c4b4-4bc5-8208-359681e17c53","Type":"ContainerStarted","Data":"a83fb32d6e184cf7bd990a9a6c5753eb6eb1ad8f48bc5356451351a4959404fb"} Sep 09 02:17:29.810772 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:29.810682 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-plmhn" Sep 09 02:17:29.829308 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:29.829262 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-plmhn" podStartSLOduration=2.958505444 podStartE2EDuration="4.829248019s" totalImagesPullingTime="1.870742575s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:17:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:17:29.828138014 +0000 UTC m=+409.203169445" watchObservedRunningTime="2026-09-09 02:17:29.829248019 +0000 UTC m=+409.204279426" Sep 09 02:17:29.878873 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:29.878837 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8ea7a2f1-03b4-4665-8286-5ef11d4eb9df-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-mfs22\" (UID: \"8ea7a2f1-03b4-4665-8286-5ef11d4eb9df\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-mfs22" Sep 09 02:17:29.878873 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:29.878871 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8gnl\" (UniqueName: \"kubernetes.io/projected/8ea7a2f1-03b4-4665-8286-5ef11d4eb9df-kube-api-access-d8gnl\") pod \"kuadrant-console-plugin-fbdd4f445-mfs22\" (UID: \"8ea7a2f1-03b4-4665-8286-5ef11d4eb9df\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-mfs22" Sep 09 02:17:29.879158 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:29.878906 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ea7a2f1-03b4-4665-8286-5ef11d4eb9df-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-mfs22\" (UID: \"8ea7a2f1-03b4-4665-8286-5ef11d4eb9df\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-mfs22" Sep 09 02:17:29.879556 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:29.879532 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8ea7a2f1-03b4-4665-8286-5ef11d4eb9df-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-mfs22\" (UID: \"8ea7a2f1-03b4-4665-8286-5ef11d4eb9df\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-mfs22" Sep 09 02:17:29.881160 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:29.881142 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ea7a2f1-03b4-4665-8286-5ef11d4eb9df-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-mfs22\" (UID: \"8ea7a2f1-03b4-4665-8286-5ef11d4eb9df\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-mfs22" Sep 09 02:17:29.887886 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:29.887862 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8gnl\" (UniqueName: \"kubernetes.io/projected/8ea7a2f1-03b4-4665-8286-5ef11d4eb9df-kube-api-access-d8gnl\") pod \"kuadrant-console-plugin-fbdd4f445-mfs22\" (UID: \"8ea7a2f1-03b4-4665-8286-5ef11d4eb9df\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-mfs22" Sep 09 02:17:30.016947 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:30.016830 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-mfs22" Sep 09 02:17:30.293046 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:30.292957 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-mfs22"] Sep 09 02:17:30.295896 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:17:30.295873 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8ea7a2f1_03b4_4665_8286_5ef11d4eb9df.slice/crio-3b060b3eec32f224caf09cb3d8f314134c8f4e3745a0b0c4a0c21b89fc6e5354 WatchSource:0}: Error finding container 3b060b3eec32f224caf09cb3d8f314134c8f4e3745a0b0c4a0c21b89fc6e5354: Status 404 returned error can't find the container with id 3b060b3eec32f224caf09cb3d8f314134c8f4e3745a0b0c4a0c21b89fc6e5354 Sep 09 02:17:30.815606 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:30.815562 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-mfs22" event={"ID":"8ea7a2f1-03b4-4665-8286-5ef11d4eb9df","Type":"ContainerStarted","Data":"3b060b3eec32f224caf09cb3d8f314134c8f4e3745a0b0c4a0c21b89fc6e5354"} Sep 09 02:17:31.770961 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:31.770931 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5hm9l" Sep 09 02:17:35.843699 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:35.843663 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-mfs22" event={"ID":"8ea7a2f1-03b4-4665-8286-5ef11d4eb9df","Type":"ContainerStarted","Data":"f8ac6f2fe85d14d231b8be1e0b476eb184df1e1cfb402ce173268d09a1bde8af"} Sep 09 02:17:35.861055 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:35.860993 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-mfs22" podStartSLOduration=1.8340554980000001 podStartE2EDuration="6.860973993s" totalImagesPullingTime="5.026918495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:17:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:17:35.860122472 +0000 UTC m=+415.235153882" watchObservedRunningTime="2026-09-09 02:17:35.860973993 +0000 UTC m=+415.236005402" Sep 09 02:17:38.806101 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:38.806066 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s88fz" Sep 09 02:17:39.816763 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:39.816722 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-stsv5"] Sep 09 02:17:39.819745 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:39.819723 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-stsv5" Sep 09 02:17:39.838537 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:39.838514 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-stsv5"] Sep 09 02:17:39.857186 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:39.857162 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79fw9\" (UniqueName: \"kubernetes.io/projected/e2a5dd78-63d0-4e45-ab48-f8b3deaf298d-kube-api-access-79fw9\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-stsv5\" (UID: \"e2a5dd78-63d0-4e45-ab48-f8b3deaf298d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-stsv5" Sep 09 02:17:39.857328 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:39.857204 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e2a5dd78-63d0-4e45-ab48-f8b3deaf298d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-stsv5\" (UID: \"e2a5dd78-63d0-4e45-ab48-f8b3deaf298d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-stsv5" Sep 09 02:17:39.958177 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:39.958143 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79fw9\" (UniqueName: \"kubernetes.io/projected/e2a5dd78-63d0-4e45-ab48-f8b3deaf298d-kube-api-access-79fw9\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-stsv5\" (UID: \"e2a5dd78-63d0-4e45-ab48-f8b3deaf298d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-stsv5" Sep 09 02:17:39.958347 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:39.958190 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e2a5dd78-63d0-4e45-ab48-f8b3deaf298d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-stsv5\" (UID: \"e2a5dd78-63d0-4e45-ab48-f8b3deaf298d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-stsv5" Sep 09 02:17:39.958508 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:39.958491 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e2a5dd78-63d0-4e45-ab48-f8b3deaf298d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-stsv5\" (UID: \"e2a5dd78-63d0-4e45-ab48-f8b3deaf298d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-stsv5" Sep 09 02:17:39.968897 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:39.968864 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79fw9\" (UniqueName: \"kubernetes.io/projected/e2a5dd78-63d0-4e45-ab48-f8b3deaf298d-kube-api-access-79fw9\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-stsv5\" (UID: \"e2a5dd78-63d0-4e45-ab48-f8b3deaf298d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-stsv5" Sep 09 02:17:40.129197 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:40.129109 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-stsv5" Sep 09 02:17:40.407147 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:40.407125 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-stsv5"] Sep 09 02:17:40.409701 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:17:40.409672 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2a5dd78_63d0_4e45_ab48_f8b3deaf298d.slice/crio-3bb338f486dcd1516a97eef5464f81541b797a079d2a85034af55ff20d35b7f2 WatchSource:0}: Error finding container 3bb338f486dcd1516a97eef5464f81541b797a079d2a85034af55ff20d35b7f2: Status 404 returned error can't find the container with id 3bb338f486dcd1516a97eef5464f81541b797a079d2a85034af55ff20d35b7f2 Sep 09 02:17:40.818687 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:40.818659 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-plmhn" Sep 09 02:17:40.867997 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:40.867960 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-stsv5" event={"ID":"e2a5dd78-63d0-4e45-ab48-f8b3deaf298d","Type":"ContainerStarted","Data":"bca1b7989525a70e7e9b6ba180ec00eb95a4f0ecb92e0a868b9c5ff089ab2908"} Sep 09 02:17:40.868174 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:40.868036 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-stsv5" event={"ID":"e2a5dd78-63d0-4e45-ab48-f8b3deaf298d","Type":"ContainerStarted","Data":"3bb338f486dcd1516a97eef5464f81541b797a079d2a85034af55ff20d35b7f2"} Sep 09 02:17:40.868174 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:40.868144 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-stsv5" Sep 09 02:17:40.889767 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:40.889709 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-stsv5" podStartSLOduration=1.889689111 podStartE2EDuration="1.889689111s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:17:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:17:40.888077507 +0000 UTC m=+420.263108915" watchObservedRunningTime="2026-09-09 02:17:40.889689111 +0000 UTC m=+420.264720521" Sep 09 02:17:40.920476 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:40.920441 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-stsv5"] Sep 09 02:17:40.929886 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:40.929860 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-stsv5"] Sep 09 02:17:40.943177 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:40.943150 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s88fz"] Sep 09 02:17:40.943418 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:40.943392 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s88fz" podUID="235fb8c7-2620-4830-b04d-cee7203c8232" containerName="manager" containerID="cri-o://9b4a89b5a340d58e5d13d4bc8a7de3d2ab9951dde1519bd193a2fc9319eccfbc" gracePeriod=2 Sep 09 02:17:40.956498 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:40.956473 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-89p7c"] Sep 09 02:17:40.956793 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:40.956781 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2a5dd78-63d0-4e45-ab48-f8b3deaf298d" containerName="manager" Sep 09 02:17:40.956838 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:40.956795 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2a5dd78-63d0-4e45-ab48-f8b3deaf298d" containerName="manager" Sep 09 02:17:40.956872 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:40.956850 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2a5dd78-63d0-4e45-ab48-f8b3deaf298d" containerName="manager" Sep 09 02:17:40.958745 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:40.958728 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-89p7c" Sep 09 02:17:40.965823 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:40.965804 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s88fz"] Sep 09 02:17:40.980453 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:40.980416 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-89p7c"] Sep 09 02:17:40.981947 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:40.981923 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-dljqg"] Sep 09 02:17:40.982187 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:40.982148 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dljqg" podUID="bbf56ca1-18be-45d2-b5e7-4d4a3473b2bb" containerName="manager" containerID="cri-o://c4b92ad043630b867b59a8470aecfa8531a760447cb511bb7d58f2a5da21b3de" gracePeriod=2 Sep 09 02:17:40.985859 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:17:40.985774 2561 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"235fb8c7-2620-4830-b04d-cee7203c8232\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-76f8f686bf-s88fz\": pods \"kuadrant-operator-controller-manager-76f8f686bf-s88fz\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s88fz" Sep 09 02:17:40.987949 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:40.987927 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-dljqg"] Sep 09 02:17:40.991629 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:17:40.991601 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-dljqg\" is forbidden: User \"system:node:ip-10-0-128-218.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-218.ec2.internal' and this object" podUID="bbf56ca1-18be-45d2-b5e7-4d4a3473b2bb" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dljqg" Sep 09 02:17:40.993387 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:17:40.993362 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-s88fz\" is forbidden: User \"system:node:ip-10-0-128-218.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-218.ec2.internal' and this object" podUID="235fb8c7-2620-4830-b04d-cee7203c8232" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s88fz" Sep 09 02:17:41.026346 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.026307 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-bhxf5"] Sep 09 02:17:41.026803 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.026781 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbf56ca1-18be-45d2-b5e7-4d4a3473b2bb" containerName="manager" Sep 09 02:17:41.026893 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.026806 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbf56ca1-18be-45d2-b5e7-4d4a3473b2bb" containerName="manager" Sep 09 02:17:41.026893 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.026837 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="235fb8c7-2620-4830-b04d-cee7203c8232" containerName="manager" Sep 09 02:17:41.026893 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.026846 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="235fb8c7-2620-4830-b04d-cee7203c8232" containerName="manager" Sep 09 02:17:41.027063 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.026922 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbf56ca1-18be-45d2-b5e7-4d4a3473b2bb" containerName="manager" Sep 09 02:17:41.027063 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.026939 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="235fb8c7-2620-4830-b04d-cee7203c8232" containerName="manager" Sep 09 02:17:41.029256 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.029234 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bhxf5" Sep 09 02:17:41.050752 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.050722 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-bhxf5"] Sep 09 02:17:41.064302 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:17:41.064265 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-dljqg\" is forbidden: User \"system:node:ip-10-0-128-218.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-218.ec2.internal' and this object" podUID="bbf56ca1-18be-45d2-b5e7-4d4a3473b2bb" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dljqg" Sep 09 02:17:41.067323 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.067224 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/97255381-76e2-4489-b23a-a15db802179d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-89p7c\" (UID: \"97255381-76e2-4489-b23a-a15db802179d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-89p7c" Sep 09 02:17:41.067323 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.067262 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qs2b\" (UniqueName: \"kubernetes.io/projected/a7b1d8fd-ca1d-40f2-ab83-9a1621c3421c-kube-api-access-2qs2b\") pod \"limitador-operator-controller-manager-74495c69f8-bhxf5\" (UID: \"a7b1d8fd-ca1d-40f2-ab83-9a1621c3421c\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bhxf5" Sep 09 02:17:41.067552 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.067351 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbztf\" (UniqueName: \"kubernetes.io/projected/97255381-76e2-4489-b23a-a15db802179d-kube-api-access-hbztf\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-89p7c\" (UID: \"97255381-76e2-4489-b23a-a15db802179d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-89p7c" Sep 09 02:17:41.069580 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:17:41.069523 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-s88fz\" is forbidden: User \"system:node:ip-10-0-128-218.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-218.ec2.internal' and this object" podUID="235fb8c7-2620-4830-b04d-cee7203c8232" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s88fz" Sep 09 02:17:41.110968 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:17:41.110930 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-dljqg\" is forbidden: User \"system:node:ip-10-0-128-218.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-218.ec2.internal' and this object" podUID="bbf56ca1-18be-45d2-b5e7-4d4a3473b2bb" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dljqg" Sep 09 02:17:41.113598 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:17:41.113548 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-s88fz\" is forbidden: User \"system:node:ip-10-0-128-218.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-218.ec2.internal' and this object" podUID="235fb8c7-2620-4830-b04d-cee7203c8232" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s88fz" Sep 09 02:17:41.168558 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.168527 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/97255381-76e2-4489-b23a-a15db802179d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-89p7c\" (UID: \"97255381-76e2-4489-b23a-a15db802179d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-89p7c" Sep 09 02:17:41.168558 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.168562 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qs2b\" (UniqueName: \"kubernetes.io/projected/a7b1d8fd-ca1d-40f2-ab83-9a1621c3421c-kube-api-access-2qs2b\") pod \"limitador-operator-controller-manager-74495c69f8-bhxf5\" (UID: \"a7b1d8fd-ca1d-40f2-ab83-9a1621c3421c\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bhxf5" Sep 09 02:17:41.168771 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.168606 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbztf\" (UniqueName: \"kubernetes.io/projected/97255381-76e2-4489-b23a-a15db802179d-kube-api-access-hbztf\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-89p7c\" (UID: \"97255381-76e2-4489-b23a-a15db802179d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-89p7c" Sep 09 02:17:41.169000 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.168973 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/97255381-76e2-4489-b23a-a15db802179d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-89p7c\" (UID: \"97255381-76e2-4489-b23a-a15db802179d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-89p7c" Sep 09 02:17:41.185784 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.185758 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qs2b\" (UniqueName: \"kubernetes.io/projected/a7b1d8fd-ca1d-40f2-ab83-9a1621c3421c-kube-api-access-2qs2b\") pod \"limitador-operator-controller-manager-74495c69f8-bhxf5\" (UID: \"a7b1d8fd-ca1d-40f2-ab83-9a1621c3421c\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bhxf5" Sep 09 02:17:41.185882 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.185764 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbztf\" (UniqueName: \"kubernetes.io/projected/97255381-76e2-4489-b23a-a15db802179d-kube-api-access-hbztf\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-89p7c\" (UID: \"97255381-76e2-4489-b23a-a15db802179d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-89p7c" Sep 09 02:17:41.196047 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.195998 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s88fz" Sep 09 02:17:41.213908 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.213890 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dljqg" Sep 09 02:17:41.269236 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.269202 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sfhwc\" (UniqueName: \"kubernetes.io/projected/235fb8c7-2620-4830-b04d-cee7203c8232-kube-api-access-sfhwc\") pod \"235fb8c7-2620-4830-b04d-cee7203c8232\" (UID: \"235fb8c7-2620-4830-b04d-cee7203c8232\") " Sep 09 02:17:41.269359 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.269258 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/235fb8c7-2620-4830-b04d-cee7203c8232-extensions-socket-volume\") pod \"235fb8c7-2620-4830-b04d-cee7203c8232\" (UID: \"235fb8c7-2620-4830-b04d-cee7203c8232\") " Sep 09 02:17:41.269359 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.269334 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5pzx\" (UniqueName: \"kubernetes.io/projected/bbf56ca1-18be-45d2-b5e7-4d4a3473b2bb-kube-api-access-z5pzx\") pod \"bbf56ca1-18be-45d2-b5e7-4d4a3473b2bb\" (UID: \"bbf56ca1-18be-45d2-b5e7-4d4a3473b2bb\") " Sep 09 02:17:41.269746 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.269710 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/235fb8c7-2620-4830-b04d-cee7203c8232-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "235fb8c7-2620-4830-b04d-cee7203c8232" (UID: "235fb8c7-2620-4830-b04d-cee7203c8232"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 02:17:41.271381 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.271354 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/235fb8c7-2620-4830-b04d-cee7203c8232-kube-api-access-sfhwc" (OuterVolumeSpecName: "kube-api-access-sfhwc") pod "235fb8c7-2620-4830-b04d-cee7203c8232" (UID: "235fb8c7-2620-4830-b04d-cee7203c8232"). InnerVolumeSpecName "kube-api-access-sfhwc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 02:17:41.271473 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.271422 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbf56ca1-18be-45d2-b5e7-4d4a3473b2bb-kube-api-access-z5pzx" (OuterVolumeSpecName: "kube-api-access-z5pzx") pod "bbf56ca1-18be-45d2-b5e7-4d4a3473b2bb" (UID: "bbf56ca1-18be-45d2-b5e7-4d4a3473b2bb"). InnerVolumeSpecName "kube-api-access-z5pzx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 02:17:41.370753 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.370675 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfhwc\" (UniqueName: \"kubernetes.io/projected/235fb8c7-2620-4830-b04d-cee7203c8232-kube-api-access-sfhwc\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:17:41.370753 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.370701 2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/235fb8c7-2620-4830-b04d-cee7203c8232-extensions-socket-volume\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:17:41.370753 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.370710 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5pzx\" (UniqueName: \"kubernetes.io/projected/bbf56ca1-18be-45d2-b5e7-4d4a3473b2bb-kube-api-access-z5pzx\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:17:41.372620 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.372594 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-89p7c" Sep 09 02:17:41.378756 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.378738 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bhxf5" Sep 09 02:17:41.655714 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.655642 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-89p7c"] Sep 09 02:17:41.659543 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:17:41.659516 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97255381_76e2_4489_b23a_a15db802179d.slice/crio-70425083801e01af7e02ac69339fbd9b2e34547f5f49ee04742e1f148bf9ac18 WatchSource:0}: Error finding container 70425083801e01af7e02ac69339fbd9b2e34547f5f49ee04742e1f148bf9ac18: Status 404 returned error can't find the container with id 70425083801e01af7e02ac69339fbd9b2e34547f5f49ee04742e1f148bf9ac18 Sep 09 02:17:41.661191 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.661168 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-bhxf5"] Sep 09 02:17:41.662705 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:17:41.662682 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7b1d8fd_ca1d_40f2_ab83_9a1621c3421c.slice/crio-4f06aab4e23373f60a66f7bce286a548e1f7397497bd998244d32858f751cd9c WatchSource:0}: Error finding container 4f06aab4e23373f60a66f7bce286a548e1f7397497bd998244d32858f751cd9c: Status 404 returned error can't find the container with id 4f06aab4e23373f60a66f7bce286a548e1f7397497bd998244d32858f751cd9c Sep 09 02:17:41.874395 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.874361 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bhxf5" event={"ID":"a7b1d8fd-ca1d-40f2-ab83-9a1621c3421c","Type":"ContainerStarted","Data":"e4d019c096b794b3572c7c2ae5e9afea6e67946a1f0b60f44553c59487f1d11a"} Sep 09 02:17:41.874803 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.874422 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bhxf5" event={"ID":"a7b1d8fd-ca1d-40f2-ab83-9a1621c3421c","Type":"ContainerStarted","Data":"4f06aab4e23373f60a66f7bce286a548e1f7397497bd998244d32858f751cd9c"} Sep 09 02:17:41.874803 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.874449 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bhxf5" Sep 09 02:17:41.876090 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.876065 2561 generic.go:356] "Generic (PLEG): container finished" podID="235fb8c7-2620-4830-b04d-cee7203c8232" containerID="9b4a89b5a340d58e5d13d4bc8a7de3d2ab9951dde1519bd193a2fc9319eccfbc" exitCode=0 Sep 09 02:17:41.876419 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.876109 2561 scope.go:117] "RemoveContainer" containerID="9b4a89b5a340d58e5d13d4bc8a7de3d2ab9951dde1519bd193a2fc9319eccfbc" Sep 09 02:17:41.876419 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.876110 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s88fz" Sep 09 02:17:41.877703 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.877673 2561 generic.go:356] "Generic (PLEG): container finished" podID="bbf56ca1-18be-45d2-b5e7-4d4a3473b2bb" containerID="c4b92ad043630b867b59a8470aecfa8531a760447cb511bb7d58f2a5da21b3de" exitCode=0 Sep 09 02:17:41.877838 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.877725 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dljqg" Sep 09 02:17:41.879795 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.879770 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-89p7c" event={"ID":"97255381-76e2-4489-b23a-a15db802179d","Type":"ContainerStarted","Data":"977e7818f3c3175011900416dfa8bf7bfa9cdb7be13158104936305b5801a15a"} Sep 09 02:17:41.879921 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.879829 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-89p7c" event={"ID":"97255381-76e2-4489-b23a-a15db802179d","Type":"ContainerStarted","Data":"70425083801e01af7e02ac69339fbd9b2e34547f5f49ee04742e1f148bf9ac18"} Sep 09 02:17:41.879921 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:17:41.879894 2561 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-stsv5" container="manager" Sep 09 02:17:41.885596 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.885542 2561 scope.go:117] "RemoveContainer" containerID="9b4a89b5a340d58e5d13d4bc8a7de3d2ab9951dde1519bd193a2fc9319eccfbc" Sep 09 02:17:41.885980 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:17:41.885855 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b4a89b5a340d58e5d13d4bc8a7de3d2ab9951dde1519bd193a2fc9319eccfbc\": container with ID starting with 9b4a89b5a340d58e5d13d4bc8a7de3d2ab9951dde1519bd193a2fc9319eccfbc not found: ID does not exist" containerID="9b4a89b5a340d58e5d13d4bc8a7de3d2ab9951dde1519bd193a2fc9319eccfbc" Sep 09 02:17:41.885980 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.885879 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b4a89b5a340d58e5d13d4bc8a7de3d2ab9951dde1519bd193a2fc9319eccfbc"} err="failed to get container status \"9b4a89b5a340d58e5d13d4bc8a7de3d2ab9951dde1519bd193a2fc9319eccfbc\": rpc error: code = NotFound desc = could not find container \"9b4a89b5a340d58e5d13d4bc8a7de3d2ab9951dde1519bd193a2fc9319eccfbc\": container with ID starting with 9b4a89b5a340d58e5d13d4bc8a7de3d2ab9951dde1519bd193a2fc9319eccfbc not found: ID does not exist" Sep 09 02:17:41.885980 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.885895 2561 scope.go:117] "RemoveContainer" containerID="c4b92ad043630b867b59a8470aecfa8531a760447cb511bb7d58f2a5da21b3de" Sep 09 02:17:41.891853 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.891788 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bhxf5" podStartSLOduration=1.8917702969999999 podStartE2EDuration="1.891770297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:17:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:17:41.891483552 +0000 UTC m=+421.266514961" watchObservedRunningTime="2026-09-09 02:17:41.891770297 +0000 UTC m=+421.266801708" Sep 09 02:17:41.897623 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.897585 2561 scope.go:117] "RemoveContainer" containerID="c4b92ad043630b867b59a8470aecfa8531a760447cb511bb7d58f2a5da21b3de" Sep 09 02:17:41.898227 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:17:41.898205 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4b92ad043630b867b59a8470aecfa8531a760447cb511bb7d58f2a5da21b3de\": container with ID starting with c4b92ad043630b867b59a8470aecfa8531a760447cb511bb7d58f2a5da21b3de not found: ID does not exist" containerID="c4b92ad043630b867b59a8470aecfa8531a760447cb511bb7d58f2a5da21b3de" Sep 09 02:17:41.898295 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.898233 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4b92ad043630b867b59a8470aecfa8531a760447cb511bb7d58f2a5da21b3de"} err="failed to get container status \"c4b92ad043630b867b59a8470aecfa8531a760447cb511bb7d58f2a5da21b3de\": rpc error: code = NotFound desc = could not find container \"c4b92ad043630b867b59a8470aecfa8531a760447cb511bb7d58f2a5da21b3de\": container with ID starting with c4b92ad043630b867b59a8470aecfa8531a760447cb511bb7d58f2a5da21b3de not found: ID does not exist" Sep 09 02:17:41.921525 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:41.921483 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-89p7c" podStartSLOduration=1.921469015 podStartE2EDuration="1.921469015s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:17:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:17:41.917714545 +0000 UTC m=+421.292745955" watchObservedRunningTime="2026-09-09 02:17:41.921469015 +0000 UTC m=+421.296500424" Sep 09 02:17:42.888293 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:42.888251 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-stsv5" podUID="e2a5dd78-63d0-4e45-ab48-f8b3deaf298d" containerName="manager" containerID="cri-o://bca1b7989525a70e7e9b6ba180ec00eb95a4f0ecb92e0a868b9c5ff089ab2908" gracePeriod=2 Sep 09 02:17:42.888769 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:42.888572 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-89p7c" Sep 09 02:17:43.111774 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:43.111732 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="235fb8c7-2620-4830-b04d-cee7203c8232" path="/var/lib/kubelet/pods/235fb8c7-2620-4830-b04d-cee7203c8232/volumes" Sep 09 02:17:43.112159 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:43.112141 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbf56ca1-18be-45d2-b5e7-4d4a3473b2bb" path="/var/lib/kubelet/pods/bbf56ca1-18be-45d2-b5e7-4d4a3473b2bb/volumes" Sep 09 02:17:43.138439 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:43.138385 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-stsv5" Sep 09 02:17:43.185239 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:43.185210 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e2a5dd78-63d0-4e45-ab48-f8b3deaf298d-extensions-socket-volume\") pod \"e2a5dd78-63d0-4e45-ab48-f8b3deaf298d\" (UID: \"e2a5dd78-63d0-4e45-ab48-f8b3deaf298d\") " Sep 09 02:17:43.185374 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:43.185334 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79fw9\" (UniqueName: \"kubernetes.io/projected/e2a5dd78-63d0-4e45-ab48-f8b3deaf298d-kube-api-access-79fw9\") pod \"e2a5dd78-63d0-4e45-ab48-f8b3deaf298d\" (UID: \"e2a5dd78-63d0-4e45-ab48-f8b3deaf298d\") " Sep 09 02:17:43.185605 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:43.185578 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2a5dd78-63d0-4e45-ab48-f8b3deaf298d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e2a5dd78-63d0-4e45-ab48-f8b3deaf298d" (UID: "e2a5dd78-63d0-4e45-ab48-f8b3deaf298d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 02:17:43.187478 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:43.187451 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2a5dd78-63d0-4e45-ab48-f8b3deaf298d-kube-api-access-79fw9" (OuterVolumeSpecName: "kube-api-access-79fw9") pod "e2a5dd78-63d0-4e45-ab48-f8b3deaf298d" (UID: "e2a5dd78-63d0-4e45-ab48-f8b3deaf298d"). InnerVolumeSpecName "kube-api-access-79fw9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 02:17:43.286132 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:43.286101 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79fw9\" (UniqueName: \"kubernetes.io/projected/e2a5dd78-63d0-4e45-ab48-f8b3deaf298d-kube-api-access-79fw9\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:17:43.286132 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:43.286125 2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e2a5dd78-63d0-4e45-ab48-f8b3deaf298d-extensions-socket-volume\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:17:43.893665 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:43.893629 2561 generic.go:356] "Generic (PLEG): container finished" podID="e2a5dd78-63d0-4e45-ab48-f8b3deaf298d" containerID="bca1b7989525a70e7e9b6ba180ec00eb95a4f0ecb92e0a868b9c5ff089ab2908" exitCode=0 Sep 09 02:17:43.894165 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:43.893675 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-stsv5" Sep 09 02:17:43.894165 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:43.893732 2561 scope.go:117] "RemoveContainer" containerID="bca1b7989525a70e7e9b6ba180ec00eb95a4f0ecb92e0a868b9c5ff089ab2908" Sep 09 02:17:43.902047 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:43.902028 2561 scope.go:117] "RemoveContainer" containerID="bca1b7989525a70e7e9b6ba180ec00eb95a4f0ecb92e0a868b9c5ff089ab2908" Sep 09 02:17:43.902322 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:17:43.902305 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bca1b7989525a70e7e9b6ba180ec00eb95a4f0ecb92e0a868b9c5ff089ab2908\": container with ID starting with bca1b7989525a70e7e9b6ba180ec00eb95a4f0ecb92e0a868b9c5ff089ab2908 not found: ID does not exist" containerID="bca1b7989525a70e7e9b6ba180ec00eb95a4f0ecb92e0a868b9c5ff089ab2908" Sep 09 02:17:43.902392 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:43.902334 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bca1b7989525a70e7e9b6ba180ec00eb95a4f0ecb92e0a868b9c5ff089ab2908"} err="failed to get container status \"bca1b7989525a70e7e9b6ba180ec00eb95a4f0ecb92e0a868b9c5ff089ab2908\": rpc error: code = NotFound desc = could not find container \"bca1b7989525a70e7e9b6ba180ec00eb95a4f0ecb92e0a868b9c5ff089ab2908\": container with ID starting with bca1b7989525a70e7e9b6ba180ec00eb95a4f0ecb92e0a868b9c5ff089ab2908 not found: ID does not exist" Sep 09 02:17:43.903532 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:17:43.903510 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-stsv5\" is forbidden: User \"system:node:ip-10-0-128-218.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-218.ec2.internal' and this object" podUID="e2a5dd78-63d0-4e45-ab48-f8b3deaf298d" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-stsv5" Sep 09 02:17:45.110402 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:45.110368 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2a5dd78-63d0-4e45-ab48-f8b3deaf298d" path="/var/lib/kubelet/pods/e2a5dd78-63d0-4e45-ab48-f8b3deaf298d/volumes" Sep 09 02:17:52.890036 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:52.889982 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bhxf5" Sep 09 02:17:53.895681 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:53.895650 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-89p7c" Sep 09 02:17:54.141649 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:54.141612 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-npvzz"] Sep 09 02:17:54.300543 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:54.300509 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-npvzz"] Sep 09 02:17:54.300715 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:54.300621 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-npvzz" Sep 09 02:17:54.377508 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:54.377471 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/917231e6-d141-4de0-9f8c-f5c648ddaba1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-npvzz\" (UID: \"917231e6-d141-4de0-9f8c-f5c648ddaba1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-npvzz" Sep 09 02:17:54.377677 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:54.377576 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhdcm\" (UniqueName: \"kubernetes.io/projected/917231e6-d141-4de0-9f8c-f5c648ddaba1-kube-api-access-rhdcm\") pod \"kuadrant-operator-controller-manager-5769f99d9f-npvzz\" (UID: \"917231e6-d141-4de0-9f8c-f5c648ddaba1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-npvzz" Sep 09 02:17:54.478114 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:54.477989 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/917231e6-d141-4de0-9f8c-f5c648ddaba1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-npvzz\" (UID: \"917231e6-d141-4de0-9f8c-f5c648ddaba1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-npvzz" Sep 09 02:17:54.478114 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:54.478112 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhdcm\" (UniqueName: \"kubernetes.io/projected/917231e6-d141-4de0-9f8c-f5c648ddaba1-kube-api-access-rhdcm\") pod \"kuadrant-operator-controller-manager-5769f99d9f-npvzz\" (UID: \"917231e6-d141-4de0-9f8c-f5c648ddaba1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-npvzz" Sep 09 02:17:54.478427 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:54.478403 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/917231e6-d141-4de0-9f8c-f5c648ddaba1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-npvzz\" (UID: \"917231e6-d141-4de0-9f8c-f5c648ddaba1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-npvzz" Sep 09 02:17:54.487560 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:54.487529 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhdcm\" (UniqueName: \"kubernetes.io/projected/917231e6-d141-4de0-9f8c-f5c648ddaba1-kube-api-access-rhdcm\") pod \"kuadrant-operator-controller-manager-5769f99d9f-npvzz\" (UID: \"917231e6-d141-4de0-9f8c-f5c648ddaba1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-npvzz" Sep 09 02:17:54.610827 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:54.610789 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-npvzz" Sep 09 02:17:54.928771 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:54.928736 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-npvzz"] Sep 09 02:17:54.933334 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:17:54.933306 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod917231e6_d141_4de0_9f8c_f5c648ddaba1.slice/crio-6aa4dcd87b71a6b2baf096cc6334fa0f2c41c7d181787b3f20631a7961c3eecd WatchSource:0}: Error finding container 6aa4dcd87b71a6b2baf096cc6334fa0f2c41c7d181787b3f20631a7961c3eecd: Status 404 returned error can't find the container with id 6aa4dcd87b71a6b2baf096cc6334fa0f2c41c7d181787b3f20631a7961c3eecd Sep 09 02:17:54.943657 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:54.943631 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-npvzz" event={"ID":"917231e6-d141-4de0-9f8c-f5c648ddaba1","Type":"ContainerStarted","Data":"6aa4dcd87b71a6b2baf096cc6334fa0f2c41c7d181787b3f20631a7961c3eecd"} Sep 09 02:17:55.951030 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:55.950975 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-npvzz" event={"ID":"917231e6-d141-4de0-9f8c-f5c648ddaba1","Type":"ContainerStarted","Data":"f18f50bb4da7a1b51bb1572c04337387d9386bbc052d33f23f014a562a6dc11f"} Sep 09 02:17:55.951488 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:55.951065 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-npvzz" Sep 09 02:17:55.971844 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:17:55.971797 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-npvzz" podStartSLOduration=1.9717820750000001 podStartE2EDuration="1.971782075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:17:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:17:55.970939225 +0000 UTC m=+435.345970633" watchObservedRunningTime="2026-09-09 02:17:55.971782075 +0000 UTC m=+435.346813485" Sep 09 02:18:06.958450 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:06.958377 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-npvzz" Sep 09 02:18:07.010390 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:07.010349 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-89p7c"] Sep 09 02:18:07.010707 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:07.010676 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-89p7c" podUID="97255381-76e2-4489-b23a-a15db802179d" containerName="manager" containerID="cri-o://977e7818f3c3175011900416dfa8bf7bfa9cdb7be13158104936305b5801a15a" gracePeriod=10 Sep 09 02:18:07.248832 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:07.248807 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-89p7c" Sep 09 02:18:07.390575 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:07.390537 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/97255381-76e2-4489-b23a-a15db802179d-extensions-socket-volume\") pod \"97255381-76e2-4489-b23a-a15db802179d\" (UID: \"97255381-76e2-4489-b23a-a15db802179d\") " Sep 09 02:18:07.390743 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:07.390602 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hbztf\" (UniqueName: \"kubernetes.io/projected/97255381-76e2-4489-b23a-a15db802179d-kube-api-access-hbztf\") pod \"97255381-76e2-4489-b23a-a15db802179d\" (UID: \"97255381-76e2-4489-b23a-a15db802179d\") " Sep 09 02:18:07.390943 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:07.390919 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97255381-76e2-4489-b23a-a15db802179d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "97255381-76e2-4489-b23a-a15db802179d" (UID: "97255381-76e2-4489-b23a-a15db802179d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 02:18:07.392761 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:07.392736 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97255381-76e2-4489-b23a-a15db802179d-kube-api-access-hbztf" (OuterVolumeSpecName: "kube-api-access-hbztf") pod "97255381-76e2-4489-b23a-a15db802179d" (UID: "97255381-76e2-4489-b23a-a15db802179d"). InnerVolumeSpecName "kube-api-access-hbztf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 02:18:07.491601 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:07.491508 2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/97255381-76e2-4489-b23a-a15db802179d-extensions-socket-volume\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:18:07.491601 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:07.491544 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hbztf\" (UniqueName: \"kubernetes.io/projected/97255381-76e2-4489-b23a-a15db802179d-kube-api-access-hbztf\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:18:08.009847 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:08.009810 2561 generic.go:356] "Generic (PLEG): container finished" podID="97255381-76e2-4489-b23a-a15db802179d" containerID="977e7818f3c3175011900416dfa8bf7bfa9cdb7be13158104936305b5801a15a" exitCode=0 Sep 09 02:18:08.010254 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:08.009851 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-89p7c" Sep 09 02:18:08.010254 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:08.009899 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-89p7c" event={"ID":"97255381-76e2-4489-b23a-a15db802179d","Type":"ContainerDied","Data":"977e7818f3c3175011900416dfa8bf7bfa9cdb7be13158104936305b5801a15a"} Sep 09 02:18:08.010254 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:08.009949 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-89p7c" event={"ID":"97255381-76e2-4489-b23a-a15db802179d","Type":"ContainerDied","Data":"70425083801e01af7e02ac69339fbd9b2e34547f5f49ee04742e1f148bf9ac18"} Sep 09 02:18:08.010254 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:08.009971 2561 scope.go:117] "RemoveContainer" containerID="977e7818f3c3175011900416dfa8bf7bfa9cdb7be13158104936305b5801a15a" Sep 09 02:18:08.018954 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:08.018791 2561 scope.go:117] "RemoveContainer" containerID="977e7818f3c3175011900416dfa8bf7bfa9cdb7be13158104936305b5801a15a" Sep 09 02:18:08.019137 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:18:08.019117 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"977e7818f3c3175011900416dfa8bf7bfa9cdb7be13158104936305b5801a15a\": container with ID starting with 977e7818f3c3175011900416dfa8bf7bfa9cdb7be13158104936305b5801a15a not found: ID does not exist" containerID="977e7818f3c3175011900416dfa8bf7bfa9cdb7be13158104936305b5801a15a" Sep 09 02:18:08.019193 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:08.019147 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"977e7818f3c3175011900416dfa8bf7bfa9cdb7be13158104936305b5801a15a"} err="failed to get container status \"977e7818f3c3175011900416dfa8bf7bfa9cdb7be13158104936305b5801a15a\": rpc error: code = NotFound desc = could not find container \"977e7818f3c3175011900416dfa8bf7bfa9cdb7be13158104936305b5801a15a\": container with ID starting with 977e7818f3c3175011900416dfa8bf7bfa9cdb7be13158104936305b5801a15a not found: ID does not exist" Sep 09 02:18:08.034483 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:08.034450 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-89p7c"] Sep 09 02:18:08.041392 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:08.041363 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-89p7c"] Sep 09 02:18:09.109990 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:09.109959 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97255381-76e2-4489-b23a-a15db802179d" path="/var/lib/kubelet/pods/97255381-76e2-4489-b23a-a15db802179d/volumes" Sep 09 02:18:10.483717 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:10.483682 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh"] Sep 09 02:18:10.484189 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:10.484173 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97255381-76e2-4489-b23a-a15db802179d" containerName="manager" Sep 09 02:18:10.484248 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:10.484192 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="97255381-76e2-4489-b23a-a15db802179d" containerName="manager" Sep 09 02:18:10.484283 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:10.484278 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="97255381-76e2-4489-b23a-a15db802179d" containerName="manager" Sep 09 02:18:10.487369 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:10.487349 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" Sep 09 02:18:10.490306 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:10.490285 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-s6s5z\"" Sep 09 02:18:10.490431 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:10.490402 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 09 02:18:10.509629 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:10.509602 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh"] Sep 09 02:18:10.618336 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:10.618294 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/adc17dae-c69d-4f6a-a944-c7bda3c3f681-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n2frh\" (UID: \"adc17dae-c69d-4f6a-a944-c7bda3c3f681\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" Sep 09 02:18:10.618519 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:10.618387 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8c69\" (UniqueName: \"kubernetes.io/projected/adc17dae-c69d-4f6a-a944-c7bda3c3f681-kube-api-access-f8c69\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n2frh\" (UID: \"adc17dae-c69d-4f6a-a944-c7bda3c3f681\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" Sep 09 02:18:10.618519 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:10.618428 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/adc17dae-c69d-4f6a-a944-c7bda3c3f681-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n2frh\" (UID: \"adc17dae-c69d-4f6a-a944-c7bda3c3f681\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" Sep 09 02:18:10.618519 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:10.618453 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/adc17dae-c69d-4f6a-a944-c7bda3c3f681-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n2frh\" (UID: \"adc17dae-c69d-4f6a-a944-c7bda3c3f681\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" Sep 09 02:18:10.618519 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:10.618476 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/adc17dae-c69d-4f6a-a944-c7bda3c3f681-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n2frh\" (UID: \"adc17dae-c69d-4f6a-a944-c7bda3c3f681\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" Sep 09 02:18:10.618752 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:10.618526 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adc17dae-c69d-4f6a-a944-c7bda3c3f681-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n2frh\" (UID: \"adc17dae-c69d-4f6a-a944-c7bda3c3f681\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" Sep 09 02:18:10.618752 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:10.618556 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/adc17dae-c69d-4f6a-a944-c7bda3c3f681-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n2frh\" (UID: \"adc17dae-c69d-4f6a-a944-c7bda3c3f681\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" Sep 09 02:18:10.618752 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:10.618617 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/adc17dae-c69d-4f6a-a944-c7bda3c3f681-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n2frh\" (UID: \"adc17dae-c69d-4f6a-a944-c7bda3c3f681\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" Sep 09 02:18:10.618752 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:10.618678 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/adc17dae-c69d-4f6a-a944-c7bda3c3f681-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n2frh\" (UID: \"adc17dae-c69d-4f6a-a944-c7bda3c3f681\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" Sep 09 02:18:10.720132 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:10.720096 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/adc17dae-c69d-4f6a-a944-c7bda3c3f681-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n2frh\" (UID: \"adc17dae-c69d-4f6a-a944-c7bda3c3f681\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" Sep 09 02:18:10.720132 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:10.720136 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/adc17dae-c69d-4f6a-a944-c7bda3c3f681-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n2frh\" (UID: \"adc17dae-c69d-4f6a-a944-c7bda3c3f681\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" Sep 09 02:18:10.720390 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:10.720157 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/adc17dae-c69d-4f6a-a944-c7bda3c3f681-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n2frh\" (UID: \"adc17dae-c69d-4f6a-a944-c7bda3c3f681\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" Sep 09 02:18:10.720390 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:10.720178 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adc17dae-c69d-4f6a-a944-c7bda3c3f681-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n2frh\" (UID: \"adc17dae-c69d-4f6a-a944-c7bda3c3f681\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" Sep 09 02:18:10.720390 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:10.720204 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/adc17dae-c69d-4f6a-a944-c7bda3c3f681-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n2frh\" (UID: \"adc17dae-c69d-4f6a-a944-c7bda3c3f681\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" Sep 09 02:18:10.720390 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:10.720231 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/adc17dae-c69d-4f6a-a944-c7bda3c3f681-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n2frh\" (UID: \"adc17dae-c69d-4f6a-a944-c7bda3c3f681\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" Sep 09 02:18:10.720390 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:10.720263 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/adc17dae-c69d-4f6a-a944-c7bda3c3f681-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n2frh\" (UID: \"adc17dae-c69d-4f6a-a944-c7bda3c3f681\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" Sep 09 02:18:10.720390 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:10.720290 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/adc17dae-c69d-4f6a-a944-c7bda3c3f681-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n2frh\" (UID: \"adc17dae-c69d-4f6a-a944-c7bda3c3f681\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" Sep 09 02:18:10.720675 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:10.720534 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8c69\" (UniqueName: \"kubernetes.io/projected/adc17dae-c69d-4f6a-a944-c7bda3c3f681-kube-api-access-f8c69\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n2frh\" (UID: \"adc17dae-c69d-4f6a-a944-c7bda3c3f681\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" Sep 09 02:18:10.720784 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:10.720757 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/adc17dae-c69d-4f6a-a944-c7bda3c3f681-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n2frh\" (UID: \"adc17dae-c69d-4f6a-a944-c7bda3c3f681\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" Sep 09 02:18:10.720864 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:10.720792 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/adc17dae-c69d-4f6a-a944-c7bda3c3f681-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n2frh\" (UID: \"adc17dae-c69d-4f6a-a944-c7bda3c3f681\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" Sep 09 02:18:10.720915 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:10.720866 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/adc17dae-c69d-4f6a-a944-c7bda3c3f681-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n2frh\" (UID: \"adc17dae-c69d-4f6a-a944-c7bda3c3f681\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" Sep 09 02:18:10.720963 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:10.720943 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/adc17dae-c69d-4f6a-a944-c7bda3c3f681-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n2frh\" (UID: \"adc17dae-c69d-4f6a-a944-c7bda3c3f681\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" Sep 09 02:18:10.721374 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:10.721352 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/adc17dae-c69d-4f6a-a944-c7bda3c3f681-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n2frh\" (UID: \"adc17dae-c69d-4f6a-a944-c7bda3c3f681\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" Sep 09 02:18:10.722683 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:10.722663 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/adc17dae-c69d-4f6a-a944-c7bda3c3f681-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n2frh\" (UID: \"adc17dae-c69d-4f6a-a944-c7bda3c3f681\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" Sep 09 02:18:10.722885 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:10.722866 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/adc17dae-c69d-4f6a-a944-c7bda3c3f681-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n2frh\" (UID: \"adc17dae-c69d-4f6a-a944-c7bda3c3f681\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" Sep 09 02:18:10.727997 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:10.727969 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adc17dae-c69d-4f6a-a944-c7bda3c3f681-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n2frh\" (UID: \"adc17dae-c69d-4f6a-a944-c7bda3c3f681\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" Sep 09 02:18:10.728175 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:10.728157 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8c69\" (UniqueName: \"kubernetes.io/projected/adc17dae-c69d-4f6a-a944-c7bda3c3f681-kube-api-access-f8c69\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n2frh\" (UID: \"adc17dae-c69d-4f6a-a944-c7bda3c3f681\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" Sep 09 02:18:10.797273 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:10.797179 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" Sep 09 02:18:11.075882 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:11.075801 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh"] Sep 09 02:18:11.079008 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:18:11.078968 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadc17dae_c69d_4f6a_a944_c7bda3c3f681.slice/crio-ea79bd6e18aa7693df312773f66e409f154bca28b99bc01489a0c863994e5190 WatchSource:0}: Error finding container ea79bd6e18aa7693df312773f66e409f154bca28b99bc01489a0c863994e5190: Status 404 returned error can't find the container with id ea79bd6e18aa7693df312773f66e409f154bca28b99bc01489a0c863994e5190 Sep 09 02:18:12.031868 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:12.031832 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" event={"ID":"adc17dae-c69d-4f6a-a944-c7bda3c3f681","Type":"ContainerStarted","Data":"ea79bd6e18aa7693df312773f66e409f154bca28b99bc01489a0c863994e5190"} Sep 09 02:18:15.476154 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:15.476120 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 09 02:18:15.476385 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:15.476188 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 09 02:18:15.476385 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:15.476218 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 09 02:18:16.052637 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:16.052598 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" event={"ID":"adc17dae-c69d-4f6a-a944-c7bda3c3f681","Type":"ContainerStarted","Data":"9e58681d5fb664d019fc335275117cc76ea789d49e63fbeb0818baf6c88583ca"} Sep 09 02:18:16.073442 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:16.073388 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" podStartSLOduration=1.678520653 podStartE2EDuration="6.073371215s" totalImagesPullingTime="4.394850562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:18:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:18:16.071772566 +0000 UTC m=+455.446803986" watchObservedRunningTime="2026-09-09 02:18:16.073371215 +0000 UTC m=+455.448402623" Sep 09 02:18:16.797502 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:16.797455 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" Sep 09 02:18:16.802123 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:16.802094 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" Sep 09 02:18:17.057450 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:17.057358 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" Sep 09 02:18:17.058490 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:17.058470 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" Sep 09 02:18:26.909628 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:26.909591 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-sj8fq"] Sep 09 02:18:26.911871 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:26.911853 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-sj8fq" Sep 09 02:18:26.914369 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:26.914345 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Sep 09 02:18:26.922246 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:26.922220 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-sj8fq"] Sep 09 02:18:26.961351 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:26.961309 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b056598e-d6f2-49a6-9a33-8531743efc28-config-file\") pod \"limitador-limitador-8588749977-sj8fq\" (UID: \"b056598e-d6f2-49a6-9a33-8531743efc28\") " pod="kuadrant-system/limitador-limitador-8588749977-sj8fq" Sep 09 02:18:26.961351 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:26.961360 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzdpj\" (UniqueName: \"kubernetes.io/projected/b056598e-d6f2-49a6-9a33-8531743efc28-kube-api-access-tzdpj\") pod \"limitador-limitador-8588749977-sj8fq\" (UID: \"b056598e-d6f2-49a6-9a33-8531743efc28\") " pod="kuadrant-system/limitador-limitador-8588749977-sj8fq" Sep 09 02:18:27.006234 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:27.006200 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-sj8fq"] Sep 09 02:18:27.061904 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:27.061870 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzdpj\" (UniqueName: \"kubernetes.io/projected/b056598e-d6f2-49a6-9a33-8531743efc28-kube-api-access-tzdpj\") pod \"limitador-limitador-8588749977-sj8fq\" (UID: \"b056598e-d6f2-49a6-9a33-8531743efc28\") " pod="kuadrant-system/limitador-limitador-8588749977-sj8fq" Sep 09 02:18:27.062097 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:27.061969 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b056598e-d6f2-49a6-9a33-8531743efc28-config-file\") pod \"limitador-limitador-8588749977-sj8fq\" (UID: \"b056598e-d6f2-49a6-9a33-8531743efc28\") " pod="kuadrant-system/limitador-limitador-8588749977-sj8fq" Sep 09 02:18:27.062815 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:27.062792 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b056598e-d6f2-49a6-9a33-8531743efc28-config-file\") pod \"limitador-limitador-8588749977-sj8fq\" (UID: \"b056598e-d6f2-49a6-9a33-8531743efc28\") " pod="kuadrant-system/limitador-limitador-8588749977-sj8fq" Sep 09 02:18:27.070163 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:27.070137 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzdpj\" (UniqueName: \"kubernetes.io/projected/b056598e-d6f2-49a6-9a33-8531743efc28-kube-api-access-tzdpj\") pod \"limitador-limitador-8588749977-sj8fq\" (UID: \"b056598e-d6f2-49a6-9a33-8531743efc28\") " pod="kuadrant-system/limitador-limitador-8588749977-sj8fq" Sep 09 02:18:27.222477 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:27.222434 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-sj8fq" Sep 09 02:18:27.491629 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:27.491598 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-sj8fq"] Sep 09 02:18:27.493865 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:18:27.493840 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb056598e_d6f2_49a6_9a33_8531743efc28.slice/crio-12a37813b10a5402b30d70ba277bde9da60ebcda3f3560acd471c3ee57060113 WatchSource:0}: Error finding container 12a37813b10a5402b30d70ba277bde9da60ebcda3f3560acd471c3ee57060113: Status 404 returned error can't find the container with id 12a37813b10a5402b30d70ba277bde9da60ebcda3f3560acd471c3ee57060113 Sep 09 02:18:28.109762 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:28.109719 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-sj8fq" event={"ID":"b056598e-d6f2-49a6-9a33-8531743efc28","Type":"ContainerStarted","Data":"12a37813b10a5402b30d70ba277bde9da60ebcda3f3560acd471c3ee57060113"} Sep 09 02:18:29.116481 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:29.116447 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-sj8fq" event={"ID":"b056598e-d6f2-49a6-9a33-8531743efc28","Type":"ContainerStarted","Data":"c2e4994165a97993321d765a7bc45a2d41fa6ad7ee05c9ea74f4b7a0e382249f"} Sep 09 02:18:29.116865 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:29.116519 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-sj8fq" Sep 09 02:18:29.134875 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:29.134776 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-sj8fq" podStartSLOduration=1.746732478 podStartE2EDuration="3.13476055s" totalImagesPullingTime="1.388028072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:18:29.133411151 +0000 UTC m=+468.508442590" watchObservedRunningTime="2026-09-09 02:18:29.13476055 +0000 UTC m=+468.509791958" Sep 09 02:18:40.124109 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:40.124078 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-sj8fq" Sep 09 02:18:49.734948 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:49.734913 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-sj8fq"] Sep 09 02:18:49.735458 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:49.735179 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-sj8fq" podUID="b056598e-d6f2-49a6-9a33-8531743efc28" containerName="limitador" containerID="cri-o://c2e4994165a97993321d765a7bc45a2d41fa6ad7ee05c9ea74f4b7a0e382249f" gracePeriod=30 Sep 09 02:18:50.123396 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:50.123356 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="kuadrant-system/limitador-limitador-8588749977-sj8fq" podUID="b056598e-d6f2-49a6-9a33-8531743efc28" containerName="limitador" probeResult="failure" output="Get \"http://10.134.0.30:8080/status\": dial tcp 10.134.0.30:8080: connect: connection refused" Sep 09 02:18:50.218549 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:50.218513 2561 generic.go:356] "Generic (PLEG): container finished" podID="b056598e-d6f2-49a6-9a33-8531743efc28" containerID="c2e4994165a97993321d765a7bc45a2d41fa6ad7ee05c9ea74f4b7a0e382249f" exitCode=0 Sep 09 02:18:50.218687 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:50.218563 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-sj8fq" event={"ID":"b056598e-d6f2-49a6-9a33-8531743efc28","Type":"ContainerDied","Data":"c2e4994165a97993321d765a7bc45a2d41fa6ad7ee05c9ea74f4b7a0e382249f"} Sep 09 02:18:50.278816 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:50.278792 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-sj8fq" Sep 09 02:18:50.345962 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:50.345873 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tzdpj\" (UniqueName: \"kubernetes.io/projected/b056598e-d6f2-49a6-9a33-8531743efc28-kube-api-access-tzdpj\") pod \"b056598e-d6f2-49a6-9a33-8531743efc28\" (UID: \"b056598e-d6f2-49a6-9a33-8531743efc28\") " Sep 09 02:18:50.345962 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:50.345951 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b056598e-d6f2-49a6-9a33-8531743efc28-config-file\") pod \"b056598e-d6f2-49a6-9a33-8531743efc28\" (UID: \"b056598e-d6f2-49a6-9a33-8531743efc28\") " Sep 09 02:18:50.346389 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:50.346364 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b056598e-d6f2-49a6-9a33-8531743efc28-config-file" (OuterVolumeSpecName: "config-file") pod "b056598e-d6f2-49a6-9a33-8531743efc28" (UID: "b056598e-d6f2-49a6-9a33-8531743efc28"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 02:18:50.347964 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:50.347938 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b056598e-d6f2-49a6-9a33-8531743efc28-kube-api-access-tzdpj" (OuterVolumeSpecName: "kube-api-access-tzdpj") pod "b056598e-d6f2-49a6-9a33-8531743efc28" (UID: "b056598e-d6f2-49a6-9a33-8531743efc28"). InnerVolumeSpecName "kube-api-access-tzdpj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 02:18:50.446644 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:50.446612 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzdpj\" (UniqueName: \"kubernetes.io/projected/b056598e-d6f2-49a6-9a33-8531743efc28-kube-api-access-tzdpj\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:18:50.446644 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:50.446640 2561 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b056598e-d6f2-49a6-9a33-8531743efc28-config-file\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:18:51.224538 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:51.224502 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-sj8fq" event={"ID":"b056598e-d6f2-49a6-9a33-8531743efc28","Type":"ContainerDied","Data":"12a37813b10a5402b30d70ba277bde9da60ebcda3f3560acd471c3ee57060113"} Sep 09 02:18:51.224950 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:51.224566 2561 scope.go:117] "RemoveContainer" containerID="c2e4994165a97993321d765a7bc45a2d41fa6ad7ee05c9ea74f4b7a0e382249f" Sep 09 02:18:51.224950 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:51.224522 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-sj8fq" Sep 09 02:18:51.253616 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:51.253585 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-sj8fq"] Sep 09 02:18:51.260038 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:51.259998 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-sj8fq"] Sep 09 02:18:53.110551 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:53.110514 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b056598e-d6f2-49a6-9a33-8531743efc28" path="/var/lib/kubelet/pods/b056598e-d6f2-49a6-9a33-8531743efc28/volumes" Sep 09 02:18:54.304859 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:54.304826 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-ccbtf"] Sep 09 02:18:54.305317 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:54.305300 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b056598e-d6f2-49a6-9a33-8531743efc28" containerName="limitador" Sep 09 02:18:54.305367 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:54.305321 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b056598e-d6f2-49a6-9a33-8531743efc28" containerName="limitador" Sep 09 02:18:54.305451 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:54.305441 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b056598e-d6f2-49a6-9a33-8531743efc28" containerName="limitador" Sep 09 02:18:54.309986 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:54.309966 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ccbtf" Sep 09 02:18:54.312742 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:54.312722 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Sep 09 02:18:54.312858 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:54.312771 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 09 02:18:54.313804 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:54.313787 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-qrqsm\"" Sep 09 02:18:54.313906 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:54.313816 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Sep 09 02:18:54.316801 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:54.316777 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-ccbtf"] Sep 09 02:18:54.381845 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:54.381801 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87s7l\" (UniqueName: \"kubernetes.io/projected/28ec02cd-97b1-4007-a16c-41640fb8c7d5-kube-api-access-87s7l\") pod \"postgres-69fb4594cf-ccbtf\" (UID: \"28ec02cd-97b1-4007-a16c-41640fb8c7d5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ccbtf" Sep 09 02:18:54.381993 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:54.381862 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/28ec02cd-97b1-4007-a16c-41640fb8c7d5-data\") pod \"postgres-69fb4594cf-ccbtf\" (UID: \"28ec02cd-97b1-4007-a16c-41640fb8c7d5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ccbtf" Sep 09 02:18:54.483051 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:54.482995 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87s7l\" (UniqueName: \"kubernetes.io/projected/28ec02cd-97b1-4007-a16c-41640fb8c7d5-kube-api-access-87s7l\") pod \"postgres-69fb4594cf-ccbtf\" (UID: \"28ec02cd-97b1-4007-a16c-41640fb8c7d5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ccbtf" Sep 09 02:18:54.483229 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:54.483074 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/28ec02cd-97b1-4007-a16c-41640fb8c7d5-data\") pod \"postgres-69fb4594cf-ccbtf\" (UID: \"28ec02cd-97b1-4007-a16c-41640fb8c7d5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ccbtf" Sep 09 02:18:54.483512 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:54.483494 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/28ec02cd-97b1-4007-a16c-41640fb8c7d5-data\") pod \"postgres-69fb4594cf-ccbtf\" (UID: \"28ec02cd-97b1-4007-a16c-41640fb8c7d5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ccbtf" Sep 09 02:18:54.491072 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:54.491044 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87s7l\" (UniqueName: \"kubernetes.io/projected/28ec02cd-97b1-4007-a16c-41640fb8c7d5-kube-api-access-87s7l\") pod \"postgres-69fb4594cf-ccbtf\" (UID: \"28ec02cd-97b1-4007-a16c-41640fb8c7d5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ccbtf" Sep 09 02:18:54.621575 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:54.621497 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ccbtf" Sep 09 02:18:54.889191 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:54.889162 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-ccbtf"] Sep 09 02:18:54.891795 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:18:54.891768 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28ec02cd_97b1_4007_a16c_41640fb8c7d5.slice/crio-76c83c72011d8ed7184813f89571a55a717a9a9a155ca779891af4431f48126a WatchSource:0}: Error finding container 76c83c72011d8ed7184813f89571a55a717a9a9a155ca779891af4431f48126a: Status 404 returned error can't find the container with id 76c83c72011d8ed7184813f89571a55a717a9a9a155ca779891af4431f48126a Sep 09 02:18:55.245216 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:18:55.245175 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ccbtf" event={"ID":"28ec02cd-97b1-4007-a16c-41640fb8c7d5","Type":"ContainerStarted","Data":"76c83c72011d8ed7184813f89571a55a717a9a9a155ca779891af4431f48126a"} Sep 09 02:19:01.277264 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:01.277226 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ccbtf" event={"ID":"28ec02cd-97b1-4007-a16c-41640fb8c7d5","Type":"ContainerStarted","Data":"a2e5cb81b1c3c3496cdb85b72de856b41c0c17d9d1d2c780d7f37281e228a2d1"} Sep 09 02:19:01.277767 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:01.277366 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ccbtf" Sep 09 02:19:01.293508 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:01.293455 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ccbtf" podStartSLOduration=1.603122207 podStartE2EDuration="7.293438993s" totalImagesPullingTime="5.690316786s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:18:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:19:01.292326384 +0000 UTC m=+500.667357792" watchObservedRunningTime="2026-09-09 02:19:01.293438993 +0000 UTC m=+500.668470402" Sep 09 02:19:07.310509 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:07.310467 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ccbtf" Sep 09 02:19:14.957144 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:14.957110 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-58fc47f794-8tdq6"] Sep 09 02:19:14.961125 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:14.961047 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58fc47f794-8tdq6" Sep 09 02:19:14.963362 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:14.963332 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 09 02:19:14.963489 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:14.963419 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 09 02:19:14.963679 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:14.963659 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-4bwjv\"" Sep 09 02:19:14.964384 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:14.964363 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 09 02:19:14.975993 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:14.975972 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58fc47f794-8tdq6"] Sep 09 02:19:15.073037 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:15.068410 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6d719ca6-34d3-4d69-a421-489d0083b38b-webhook-cert\") pod \"maas-controller-58fc47f794-8tdq6\" (UID: \"6d719ca6-34d3-4d69-a421-489d0083b38b\") " pod="opendatahub/maas-controller-58fc47f794-8tdq6" Sep 09 02:19:15.073037 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:15.068477 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45nkj\" (UniqueName: \"kubernetes.io/projected/6d719ca6-34d3-4d69-a421-489d0083b38b-kube-api-access-45nkj\") pod \"maas-controller-58fc47f794-8tdq6\" (UID: \"6d719ca6-34d3-4d69-a421-489d0083b38b\") " pod="opendatahub/maas-controller-58fc47f794-8tdq6" Sep 09 02:19:15.073037 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:15.068585 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d719ca6-34d3-4d69-a421-489d0083b38b-metrics-certs\") pod \"maas-controller-58fc47f794-8tdq6\" (UID: \"6d719ca6-34d3-4d69-a421-489d0083b38b\") " pod="opendatahub/maas-controller-58fc47f794-8tdq6" Sep 09 02:19:15.141633 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:15.141602 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5fb57c948c-p5nmx"] Sep 09 02:19:15.144992 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:15.144970 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5fb57c948c-p5nmx" Sep 09 02:19:15.155955 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:15.155932 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5fb57c948c-p5nmx"] Sep 09 02:19:15.169746 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:15.169719 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d719ca6-34d3-4d69-a421-489d0083b38b-metrics-certs\") pod \"maas-controller-58fc47f794-8tdq6\" (UID: \"6d719ca6-34d3-4d69-a421-489d0083b38b\") " pod="opendatahub/maas-controller-58fc47f794-8tdq6" Sep 09 02:19:15.169874 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:15.169753 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6d719ca6-34d3-4d69-a421-489d0083b38b-webhook-cert\") pod \"maas-controller-58fc47f794-8tdq6\" (UID: \"6d719ca6-34d3-4d69-a421-489d0083b38b\") " pod="opendatahub/maas-controller-58fc47f794-8tdq6" Sep 09 02:19:15.169874 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:15.169785 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45nkj\" (UniqueName: \"kubernetes.io/projected/6d719ca6-34d3-4d69-a421-489d0083b38b-kube-api-access-45nkj\") pod \"maas-controller-58fc47f794-8tdq6\" (UID: \"6d719ca6-34d3-4d69-a421-489d0083b38b\") " pod="opendatahub/maas-controller-58fc47f794-8tdq6" Sep 09 02:19:15.169995 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:19:15.169893 2561 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Sep 09 02:19:15.169995 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:19:15.169956 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d719ca6-34d3-4d69-a421-489d0083b38b-webhook-cert podName:6d719ca6-34d3-4d69-a421-489d0083b38b nodeName:}" failed. No retries permitted until 2026-09-09 02:19:15.669937546 +0000 UTC m=+515.044968939 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/6d719ca6-34d3-4d69-a421-489d0083b38b-webhook-cert") pod "maas-controller-58fc47f794-8tdq6" (UID: "6d719ca6-34d3-4d69-a421-489d0083b38b") : secret "maas-controller-webhook-cert" not found Sep 09 02:19:15.172644 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:15.172617 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d719ca6-34d3-4d69-a421-489d0083b38b-metrics-certs\") pod \"maas-controller-58fc47f794-8tdq6\" (UID: \"6d719ca6-34d3-4d69-a421-489d0083b38b\") " pod="opendatahub/maas-controller-58fc47f794-8tdq6" Sep 09 02:19:15.180280 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:15.180254 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45nkj\" (UniqueName: \"kubernetes.io/projected/6d719ca6-34d3-4d69-a421-489d0083b38b-kube-api-access-45nkj\") pod \"maas-controller-58fc47f794-8tdq6\" (UID: \"6d719ca6-34d3-4d69-a421-489d0083b38b\") " pod="opendatahub/maas-controller-58fc47f794-8tdq6" Sep 09 02:19:15.271085 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:15.270977 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1129f7d2-8c55-464c-9f3f-c5ebca809be6-metrics-certs\") pod \"maas-controller-5fb57c948c-p5nmx\" (UID: \"1129f7d2-8c55-464c-9f3f-c5ebca809be6\") " pod="opendatahub/maas-controller-5fb57c948c-p5nmx" Sep 09 02:19:15.271085 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:15.271041 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8wvb\" (UniqueName: \"kubernetes.io/projected/1129f7d2-8c55-464c-9f3f-c5ebca809be6-kube-api-access-n8wvb\") pod \"maas-controller-5fb57c948c-p5nmx\" (UID: \"1129f7d2-8c55-464c-9f3f-c5ebca809be6\") " pod="opendatahub/maas-controller-5fb57c948c-p5nmx" Sep 09 02:19:15.271267 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:15.271150 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1129f7d2-8c55-464c-9f3f-c5ebca809be6-webhook-cert\") pod \"maas-controller-5fb57c948c-p5nmx\" (UID: \"1129f7d2-8c55-464c-9f3f-c5ebca809be6\") " pod="opendatahub/maas-controller-5fb57c948c-p5nmx" Sep 09 02:19:15.371956 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:15.371918 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1129f7d2-8c55-464c-9f3f-c5ebca809be6-metrics-certs\") pod \"maas-controller-5fb57c948c-p5nmx\" (UID: \"1129f7d2-8c55-464c-9f3f-c5ebca809be6\") " pod="opendatahub/maas-controller-5fb57c948c-p5nmx" Sep 09 02:19:15.372163 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:15.371963 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8wvb\" (UniqueName: \"kubernetes.io/projected/1129f7d2-8c55-464c-9f3f-c5ebca809be6-kube-api-access-n8wvb\") pod \"maas-controller-5fb57c948c-p5nmx\" (UID: \"1129f7d2-8c55-464c-9f3f-c5ebca809be6\") " pod="opendatahub/maas-controller-5fb57c948c-p5nmx" Sep 09 02:19:15.372163 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:15.372007 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1129f7d2-8c55-464c-9f3f-c5ebca809be6-webhook-cert\") pod \"maas-controller-5fb57c948c-p5nmx\" (UID: \"1129f7d2-8c55-464c-9f3f-c5ebca809be6\") " pod="opendatahub/maas-controller-5fb57c948c-p5nmx" Sep 09 02:19:15.374447 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:15.374421 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1129f7d2-8c55-464c-9f3f-c5ebca809be6-metrics-certs\") pod \"maas-controller-5fb57c948c-p5nmx\" (UID: \"1129f7d2-8c55-464c-9f3f-c5ebca809be6\") " pod="opendatahub/maas-controller-5fb57c948c-p5nmx" Sep 09 02:19:15.374539 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:15.374467 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1129f7d2-8c55-464c-9f3f-c5ebca809be6-webhook-cert\") pod \"maas-controller-5fb57c948c-p5nmx\" (UID: \"1129f7d2-8c55-464c-9f3f-c5ebca809be6\") " pod="opendatahub/maas-controller-5fb57c948c-p5nmx" Sep 09 02:19:15.381633 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:15.381613 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8wvb\" (UniqueName: \"kubernetes.io/projected/1129f7d2-8c55-464c-9f3f-c5ebca809be6-kube-api-access-n8wvb\") pod \"maas-controller-5fb57c948c-p5nmx\" (UID: \"1129f7d2-8c55-464c-9f3f-c5ebca809be6\") " pod="opendatahub/maas-controller-5fb57c948c-p5nmx" Sep 09 02:19:15.456208 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:15.456172 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5fb57c948c-p5nmx" Sep 09 02:19:15.674861 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:15.674823 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6d719ca6-34d3-4d69-a421-489d0083b38b-webhook-cert\") pod \"maas-controller-58fc47f794-8tdq6\" (UID: \"6d719ca6-34d3-4d69-a421-489d0083b38b\") " pod="opendatahub/maas-controller-58fc47f794-8tdq6" Sep 09 02:19:15.677243 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:15.677224 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6d719ca6-34d3-4d69-a421-489d0083b38b-webhook-cert\") pod \"maas-controller-58fc47f794-8tdq6\" (UID: \"6d719ca6-34d3-4d69-a421-489d0083b38b\") " pod="opendatahub/maas-controller-58fc47f794-8tdq6" Sep 09 02:19:15.729249 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:15.729216 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5fb57c948c-p5nmx"] Sep 09 02:19:15.731838 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:19:15.731809 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1129f7d2_8c55_464c_9f3f_c5ebca809be6.slice/crio-44ae9f76653add3247d1db4a375520a133830fe99693e2941fbdbef167a1db10 WatchSource:0}: Error finding container 44ae9f76653add3247d1db4a375520a133830fe99693e2941fbdbef167a1db10: Status 404 returned error can't find the container with id 44ae9f76653add3247d1db4a375520a133830fe99693e2941fbdbef167a1db10 Sep 09 02:19:15.872140 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:15.872101 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58fc47f794-8tdq6" Sep 09 02:19:16.145747 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:16.145720 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58fc47f794-8tdq6"] Sep 09 02:19:16.148218 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:19:16.148195 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d719ca6_34d3_4d69_a421_489d0083b38b.slice/crio-b55b99fcce1a496429c93c4aaddbb0fcc8e3bca41b84fc25dc0772d244efea77 WatchSource:0}: Error finding container b55b99fcce1a496429c93c4aaddbb0fcc8e3bca41b84fc25dc0772d244efea77: Status 404 returned error can't find the container with id b55b99fcce1a496429c93c4aaddbb0fcc8e3bca41b84fc25dc0772d244efea77 Sep 09 02:19:16.342958 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:16.342919 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5fb57c948c-p5nmx" event={"ID":"1129f7d2-8c55-464c-9f3f-c5ebca809be6","Type":"ContainerStarted","Data":"44ae9f76653add3247d1db4a375520a133830fe99693e2941fbdbef167a1db10"} Sep 09 02:19:16.343972 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:16.343949 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58fc47f794-8tdq6" event={"ID":"6d719ca6-34d3-4d69-a421-489d0083b38b","Type":"ContainerStarted","Data":"b55b99fcce1a496429c93c4aaddbb0fcc8e3bca41b84fc25dc0772d244efea77"} Sep 09 02:19:18.354313 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:18.354267 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5fb57c948c-p5nmx" event={"ID":"1129f7d2-8c55-464c-9f3f-c5ebca809be6","Type":"ContainerStarted","Data":"3a578433864191f3ee4c4fbe0767791d9d7466454a98b928288066287ef8e8e1"} Sep 09 02:19:18.354723 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:18.354393 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5fb57c948c-p5nmx" Sep 09 02:19:18.355783 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:18.355760 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58fc47f794-8tdq6" event={"ID":"6d719ca6-34d3-4d69-a421-489d0083b38b","Type":"ContainerStarted","Data":"118b352f4c7d96fce4a4c18ec30645aa71588772474a3ed8cb078060fc2a4c3d"} Sep 09 02:19:18.355898 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:18.355884 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-58fc47f794-8tdq6" Sep 09 02:19:18.377782 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:18.377738 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5fb57c948c-p5nmx" podStartSLOduration=1.554242054 podStartE2EDuration="3.377724499s" totalImagesPullingTime="1.823482445s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:19:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:19:18.375939832 +0000 UTC m=+517.750971242" watchObservedRunningTime="2026-09-09 02:19:18.377724499 +0000 UTC m=+517.752755907" Sep 09 02:19:18.400951 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:18.400903 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-58fc47f794-8tdq6" podStartSLOduration=2.991102632 podStartE2EDuration="4.400885692s" totalImagesPullingTime="1.40978306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:19:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:19:18.399498997 +0000 UTC m=+517.774530405" watchObservedRunningTime="2026-09-09 02:19:18.400885692 +0000 UTC m=+517.775917101" Sep 09 02:19:29.363382 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:29.363353 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5fb57c948c-p5nmx" Sep 09 02:19:29.363820 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:29.363537 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-58fc47f794-8tdq6" Sep 09 02:19:29.427885 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:29.427849 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-58fc47f794-8tdq6"] Sep 09 02:19:29.428251 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:29.428214 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-58fc47f794-8tdq6" podUID="6d719ca6-34d3-4d69-a421-489d0083b38b" containerName="manager" containerID="cri-o://118b352f4c7d96fce4a4c18ec30645aa71588772474a3ed8cb078060fc2a4c3d" gracePeriod=30 Sep 09 02:19:29.665397 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:29.665373 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58fc47f794-8tdq6" Sep 09 02:19:29.687260 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:29.687228 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6d719ca6-34d3-4d69-a421-489d0083b38b-webhook-cert\") pod \"6d719ca6-34d3-4d69-a421-489d0083b38b\" (UID: \"6d719ca6-34d3-4d69-a421-489d0083b38b\") " Sep 09 02:19:29.687406 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:29.687300 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45nkj\" (UniqueName: \"kubernetes.io/projected/6d719ca6-34d3-4d69-a421-489d0083b38b-kube-api-access-45nkj\") pod \"6d719ca6-34d3-4d69-a421-489d0083b38b\" (UID: \"6d719ca6-34d3-4d69-a421-489d0083b38b\") " Sep 09 02:19:29.687406 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:29.687341 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d719ca6-34d3-4d69-a421-489d0083b38b-metrics-certs\") pod \"6d719ca6-34d3-4d69-a421-489d0083b38b\" (UID: \"6d719ca6-34d3-4d69-a421-489d0083b38b\") " Sep 09 02:19:29.689542 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:29.689514 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d719ca6-34d3-4d69-a421-489d0083b38b-kube-api-access-45nkj" (OuterVolumeSpecName: "kube-api-access-45nkj") pod "6d719ca6-34d3-4d69-a421-489d0083b38b" (UID: "6d719ca6-34d3-4d69-a421-489d0083b38b"). InnerVolumeSpecName "kube-api-access-45nkj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 02:19:29.689834 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:29.689805 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d719ca6-34d3-4d69-a421-489d0083b38b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6d719ca6-34d3-4d69-a421-489d0083b38b" (UID: "6d719ca6-34d3-4d69-a421-489d0083b38b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 02:19:29.689939 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:29.689867 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d719ca6-34d3-4d69-a421-489d0083b38b-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6d719ca6-34d3-4d69-a421-489d0083b38b" (UID: "6d719ca6-34d3-4d69-a421-489d0083b38b"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 02:19:29.787948 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:29.787912 2561 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6d719ca6-34d3-4d69-a421-489d0083b38b-webhook-cert\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:19:29.787948 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:29.787943 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45nkj\" (UniqueName: \"kubernetes.io/projected/6d719ca6-34d3-4d69-a421-489d0083b38b-kube-api-access-45nkj\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:19:29.787948 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:29.787953 2561 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d719ca6-34d3-4d69-a421-489d0083b38b-metrics-certs\") on node \"ip-10-0-128-218.ec2.internal\" DevicePath \"\"" Sep 09 02:19:30.424035 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:30.423979 2561 generic.go:356] "Generic (PLEG): container finished" podID="6d719ca6-34d3-4d69-a421-489d0083b38b" containerID="118b352f4c7d96fce4a4c18ec30645aa71588772474a3ed8cb078060fc2a4c3d" exitCode=0 Sep 09 02:19:30.424505 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:30.424046 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58fc47f794-8tdq6" Sep 09 02:19:30.424505 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:30.424068 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58fc47f794-8tdq6" event={"ID":"6d719ca6-34d3-4d69-a421-489d0083b38b","Type":"ContainerDied","Data":"118b352f4c7d96fce4a4c18ec30645aa71588772474a3ed8cb078060fc2a4c3d"} Sep 09 02:19:30.424505 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:30.424122 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58fc47f794-8tdq6" event={"ID":"6d719ca6-34d3-4d69-a421-489d0083b38b","Type":"ContainerDied","Data":"b55b99fcce1a496429c93c4aaddbb0fcc8e3bca41b84fc25dc0772d244efea77"} Sep 09 02:19:30.424505 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:30.424145 2561 scope.go:117] "RemoveContainer" containerID="118b352f4c7d96fce4a4c18ec30645aa71588772474a3ed8cb078060fc2a4c3d" Sep 09 02:19:30.433548 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:30.433375 2561 scope.go:117] "RemoveContainer" containerID="118b352f4c7d96fce4a4c18ec30645aa71588772474a3ed8cb078060fc2a4c3d" Sep 09 02:19:30.433650 ip-10-0-128-218 kubenswrapper[2561]: E0909 02:19:30.433633 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"118b352f4c7d96fce4a4c18ec30645aa71588772474a3ed8cb078060fc2a4c3d\": container with ID starting with 118b352f4c7d96fce4a4c18ec30645aa71588772474a3ed8cb078060fc2a4c3d not found: ID does not exist" containerID="118b352f4c7d96fce4a4c18ec30645aa71588772474a3ed8cb078060fc2a4c3d" Sep 09 02:19:30.433694 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:30.433658 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"118b352f4c7d96fce4a4c18ec30645aa71588772474a3ed8cb078060fc2a4c3d"} err="failed to get container status \"118b352f4c7d96fce4a4c18ec30645aa71588772474a3ed8cb078060fc2a4c3d\": rpc error: code = NotFound desc = could not find container \"118b352f4c7d96fce4a4c18ec30645aa71588772474a3ed8cb078060fc2a4c3d\": container with ID starting with 118b352f4c7d96fce4a4c18ec30645aa71588772474a3ed8cb078060fc2a4c3d not found: ID does not exist" Sep 09 02:19:30.443849 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:30.443828 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-58fc47f794-8tdq6"] Sep 09 02:19:30.448215 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:30.448191 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-58fc47f794-8tdq6"] Sep 09 02:19:31.110845 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:31.110810 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d719ca6-34d3-4d69-a421-489d0083b38b" path="/var/lib/kubelet/pods/6d719ca6-34d3-4d69-a421-489d0083b38b/volumes" Sep 09 02:19:52.999304 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:52.999265 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6964769c6-8j58j"] Sep 09 02:19:52.999852 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:52.999824 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d719ca6-34d3-4d69-a421-489d0083b38b" containerName="manager" Sep 09 02:19:52.999852 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:52.999844 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d719ca6-34d3-4d69-a421-489d0083b38b" containerName="manager" Sep 09 02:19:52.999966 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:52.999927 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d719ca6-34d3-4d69-a421-489d0083b38b" containerName="manager" Sep 09 02:19:53.004673 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:53.004647 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6964769c6-8j58j" Sep 09 02:19:53.006782 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:53.006762 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Sep 09 02:19:53.006940 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:53.006922 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Sep 09 02:19:53.007360 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:53.007343 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8xm77\"" Sep 09 02:19:53.014416 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:53.014393 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6964769c6-8j58j"] Sep 09 02:19:53.087080 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:53.087040 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2ks8\" (UniqueName: \"kubernetes.io/projected/2eb4ecf0-eeb0-4bd9-995a-252aead22faa-kube-api-access-m2ks8\") pod \"maas-api-6964769c6-8j58j\" (UID: \"2eb4ecf0-eeb0-4bd9-995a-252aead22faa\") " pod="odh-ai-gateway-infra/maas-api-6964769c6-8j58j" Sep 09 02:19:53.087256 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:53.087146 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2eb4ecf0-eeb0-4bd9-995a-252aead22faa-maas-api-tls\") pod \"maas-api-6964769c6-8j58j\" (UID: \"2eb4ecf0-eeb0-4bd9-995a-252aead22faa\") " pod="odh-ai-gateway-infra/maas-api-6964769c6-8j58j" Sep 09 02:19:53.087256 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:53.087220 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2eb4ecf0-eeb0-4bd9-995a-252aead22faa-metrics-certs\") pod \"maas-api-6964769c6-8j58j\" (UID: \"2eb4ecf0-eeb0-4bd9-995a-252aead22faa\") " pod="odh-ai-gateway-infra/maas-api-6964769c6-8j58j" Sep 09 02:19:53.087256 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:53.087251 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2eb4ecf0-eeb0-4bd9-995a-252aead22faa-openshift-service-ca\") pod \"maas-api-6964769c6-8j58j\" (UID: \"2eb4ecf0-eeb0-4bd9-995a-252aead22faa\") " pod="odh-ai-gateway-infra/maas-api-6964769c6-8j58j" Sep 09 02:19:53.187662 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:53.187625 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2eb4ecf0-eeb0-4bd9-995a-252aead22faa-maas-api-tls\") pod \"maas-api-6964769c6-8j58j\" (UID: \"2eb4ecf0-eeb0-4bd9-995a-252aead22faa\") " pod="odh-ai-gateway-infra/maas-api-6964769c6-8j58j" Sep 09 02:19:53.187860 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:53.187727 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2eb4ecf0-eeb0-4bd9-995a-252aead22faa-metrics-certs\") pod \"maas-api-6964769c6-8j58j\" (UID: \"2eb4ecf0-eeb0-4bd9-995a-252aead22faa\") " pod="odh-ai-gateway-infra/maas-api-6964769c6-8j58j" Sep 09 02:19:53.187860 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:53.187758 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2eb4ecf0-eeb0-4bd9-995a-252aead22faa-openshift-service-ca\") pod \"maas-api-6964769c6-8j58j\" (UID: \"2eb4ecf0-eeb0-4bd9-995a-252aead22faa\") " pod="odh-ai-gateway-infra/maas-api-6964769c6-8j58j" Sep 09 02:19:53.187860 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:53.187819 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2ks8\" (UniqueName: \"kubernetes.io/projected/2eb4ecf0-eeb0-4bd9-995a-252aead22faa-kube-api-access-m2ks8\") pod \"maas-api-6964769c6-8j58j\" (UID: \"2eb4ecf0-eeb0-4bd9-995a-252aead22faa\") " pod="odh-ai-gateway-infra/maas-api-6964769c6-8j58j" Sep 09 02:19:53.188797 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:53.188769 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2eb4ecf0-eeb0-4bd9-995a-252aead22faa-openshift-service-ca\") pod \"maas-api-6964769c6-8j58j\" (UID: \"2eb4ecf0-eeb0-4bd9-995a-252aead22faa\") " pod="odh-ai-gateway-infra/maas-api-6964769c6-8j58j" Sep 09 02:19:53.190245 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:53.190216 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2eb4ecf0-eeb0-4bd9-995a-252aead22faa-maas-api-tls\") pod \"maas-api-6964769c6-8j58j\" (UID: \"2eb4ecf0-eeb0-4bd9-995a-252aead22faa\") " pod="odh-ai-gateway-infra/maas-api-6964769c6-8j58j" Sep 09 02:19:53.190346 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:53.190248 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2eb4ecf0-eeb0-4bd9-995a-252aead22faa-metrics-certs\") pod \"maas-api-6964769c6-8j58j\" (UID: \"2eb4ecf0-eeb0-4bd9-995a-252aead22faa\") " pod="odh-ai-gateway-infra/maas-api-6964769c6-8j58j" Sep 09 02:19:53.198760 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:53.198735 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2ks8\" (UniqueName: \"kubernetes.io/projected/2eb4ecf0-eeb0-4bd9-995a-252aead22faa-kube-api-access-m2ks8\") pod \"maas-api-6964769c6-8j58j\" (UID: \"2eb4ecf0-eeb0-4bd9-995a-252aead22faa\") " pod="odh-ai-gateway-infra/maas-api-6964769c6-8j58j" Sep 09 02:19:53.316560 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:53.316481 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6964769c6-8j58j" Sep 09 02:19:53.587120 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:53.587041 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-xjqc8"] Sep 09 02:19:53.591790 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:53.591772 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-xjqc8" Sep 09 02:19:53.594148 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:53.594124 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Sep 09 02:19:53.594148 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:53.594144 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-bnnzd\"" Sep 09 02:19:53.601340 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:53.600934 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6964769c6-8j58j"] Sep 09 02:19:53.602727 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:53.602705 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-xjqc8"] Sep 09 02:19:53.606001 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:19:53.605948 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2eb4ecf0_eeb0_4bd9_995a_252aead22faa.slice/crio-aedda440c92e96e0074b75da600d57b22ee3d90e25a0ffb035f9b1a99946d9bd WatchSource:0}: Error finding container aedda440c92e96e0074b75da600d57b22ee3d90e25a0ffb035f9b1a99946d9bd: Status 404 returned error can't find the container with id aedda440c92e96e0074b75da600d57b22ee3d90e25a0ffb035f9b1a99946d9bd Sep 09 02:19:53.691926 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:53.691886 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4be4d3ed-a3d7-4841-8fc2-442ec813d2b2-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-xjqc8\" (UID: \"4be4d3ed-a3d7-4841-8fc2-442ec813d2b2\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-xjqc8" Sep 09 02:19:53.692101 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:53.691931 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzvlt\" (UniqueName: \"kubernetes.io/projected/4be4d3ed-a3d7-4841-8fc2-442ec813d2b2-kube-api-access-nzvlt\") pod \"payload-pre-processing-59fbc47bf8-xjqc8\" (UID: \"4be4d3ed-a3d7-4841-8fc2-442ec813d2b2\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-xjqc8" Sep 09 02:19:53.792870 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:53.792835 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4be4d3ed-a3d7-4841-8fc2-442ec813d2b2-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-xjqc8\" (UID: \"4be4d3ed-a3d7-4841-8fc2-442ec813d2b2\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-xjqc8" Sep 09 02:19:53.792870 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:53.792873 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzvlt\" (UniqueName: \"kubernetes.io/projected/4be4d3ed-a3d7-4841-8fc2-442ec813d2b2-kube-api-access-nzvlt\") pod \"payload-pre-processing-59fbc47bf8-xjqc8\" (UID: \"4be4d3ed-a3d7-4841-8fc2-442ec813d2b2\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-xjqc8" Sep 09 02:19:53.793550 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:53.793530 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4be4d3ed-a3d7-4841-8fc2-442ec813d2b2-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-xjqc8\" (UID: \"4be4d3ed-a3d7-4841-8fc2-442ec813d2b2\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-xjqc8" Sep 09 02:19:53.801466 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:53.801444 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzvlt\" (UniqueName: \"kubernetes.io/projected/4be4d3ed-a3d7-4841-8fc2-442ec813d2b2-kube-api-access-nzvlt\") pod \"payload-pre-processing-59fbc47bf8-xjqc8\" (UID: \"4be4d3ed-a3d7-4841-8fc2-442ec813d2b2\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-xjqc8" Sep 09 02:19:53.904141 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:53.904056 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-xjqc8" Sep 09 02:19:54.188027 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:54.187989 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-xjqc8"] Sep 09 02:19:54.190328 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:19:54.190286 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4be4d3ed_a3d7_4841_8fc2_442ec813d2b2.slice/crio-9197b26c6859aa0207537eedf99d1e14ecb04c848cf948c62df6cc4da2b11139 WatchSource:0}: Error finding container 9197b26c6859aa0207537eedf99d1e14ecb04c848cf948c62df6cc4da2b11139: Status 404 returned error can't find the container with id 9197b26c6859aa0207537eedf99d1e14ecb04c848cf948c62df6cc4da2b11139 Sep 09 02:19:54.532562 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:54.532439 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-xjqc8" event={"ID":"4be4d3ed-a3d7-4841-8fc2-442ec813d2b2","Type":"ContainerStarted","Data":"9197b26c6859aa0207537eedf99d1e14ecb04c848cf948c62df6cc4da2b11139"} Sep 09 02:19:54.533780 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:54.533750 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6964769c6-8j58j" event={"ID":"2eb4ecf0-eeb0-4bd9-995a-252aead22faa","Type":"ContainerStarted","Data":"aedda440c92e96e0074b75da600d57b22ee3d90e25a0ffb035f9b1a99946d9bd"} Sep 09 02:19:57.579125 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:57.578996 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-xjqc8" event={"ID":"4be4d3ed-a3d7-4841-8fc2-442ec813d2b2","Type":"ContainerStarted","Data":"3e4963ce82f968fa7855af8431495e81833bd20d33e06ebc6d15f0bf85ac3ad9"} Sep 09 02:19:57.580114 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:57.580089 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-xjqc8" Sep 09 02:19:57.583053 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:57.582724 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6964769c6-8j58j" event={"ID":"2eb4ecf0-eeb0-4bd9-995a-252aead22faa","Type":"ContainerStarted","Data":"4d940d809e768fc793f64fb3fb28e357fb56bf56cfa097bc3aeff1b6e7c3f691"} Sep 09 02:19:57.583053 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:57.582962 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6964769c6-8j58j" Sep 09 02:19:57.595895 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:57.595662 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-xjqc8" podStartSLOduration=1.5515932129999999 podStartE2EDuration="4.595647811s" totalImagesPullingTime="3.044054598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:19:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:19:57.595591131 +0000 UTC m=+556.970622540" watchObservedRunningTime="2026-09-09 02:19:57.595647811 +0000 UTC m=+556.970679218" Sep 09 02:19:57.612031 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:19:57.611960 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6964769c6-8j58j" podStartSLOduration=2.028011785 podStartE2EDuration="5.611944593s" totalImagesPullingTime="3.583932808s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:19:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:19:57.61169789 +0000 UTC m=+556.986729299" watchObservedRunningTime="2026-09-09 02:19:57.611944593 +0000 UTC m=+556.986976000" Sep 09 02:20:02.597118 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:02.596870 2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-n2frh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": read tcp 10.134.0.2:34780->10.134.0.29:15021: read: connection reset by peer" start-of-body= Sep 09 02:20:02.597118 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:02.596940 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" podUID="adc17dae-c69d-4f6a-a944-c7bda3c3f681" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": read tcp 10.134.0.2:34780->10.134.0.29:15021: read: connection reset by peer" Sep 09 02:20:03.592857 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:03.592828 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6964769c6-8j58j" Sep 09 02:20:03.613061 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:03.613029 2561 generic.go:356] "Generic (PLEG): container finished" podID="adc17dae-c69d-4f6a-a944-c7bda3c3f681" containerID="9e58681d5fb664d019fc335275117cc76ea789d49e63fbeb0818baf6c88583ca" exitCode=0 Sep 09 02:20:03.613061 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:03.613040 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" event={"ID":"adc17dae-c69d-4f6a-a944-c7bda3c3f681","Type":"ContainerDied","Data":"9e58681d5fb664d019fc335275117cc76ea789d49e63fbeb0818baf6c88583ca"} Sep 09 02:20:03.613443 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:03.613417 2561 scope.go:117] "RemoveContainer" containerID="9e58681d5fb664d019fc335275117cc76ea789d49e63fbeb0818baf6c88583ca" Sep 09 02:20:03.614503 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:03.614478 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 09 02:20:03.614564 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:03.614540 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 09 02:20:03.614600 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:03.614578 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 09 02:20:04.619397 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:04.619365 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" event={"ID":"adc17dae-c69d-4f6a-a944-c7bda3c3f681","Type":"ContainerStarted","Data":"3d908ce65e1df940190d5aacaada5a3aa75270476cc1a74853cbb58bd41d939e"} Sep 09 02:20:04.797350 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:04.797313 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" Sep 09 02:20:04.797570 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:04.797546 2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-n2frh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body= Sep 09 02:20:04.797616 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:04.797596 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" podUID="adc17dae-c69d-4f6a-a944-c7bda3c3f681" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" Sep 09 02:20:05.798095 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:05.798058 2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-n2frh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body= Sep 09 02:20:05.798562 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:05.798122 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" podUID="adc17dae-c69d-4f6a-a944-c7bda3c3f681" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" Sep 09 02:20:06.798425 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:06.798381 2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-n2frh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body= Sep 09 02:20:06.798918 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:06.798451 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" podUID="adc17dae-c69d-4f6a-a944-c7bda3c3f681" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" Sep 09 02:20:07.798218 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:07.798184 2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-n2frh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body= Sep 09 02:20:07.798386 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:07.798240 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" podUID="adc17dae-c69d-4f6a-a944-c7bda3c3f681" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" Sep 09 02:20:08.798520 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:08.798481 2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-n2frh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body= Sep 09 02:20:08.798903 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:08.798542 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" podUID="adc17dae-c69d-4f6a-a944-c7bda3c3f681" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" Sep 09 02:20:09.594175 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:09.594144 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-xjqc8" Sep 09 02:20:09.801278 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:09.801246 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" Sep 09 02:20:09.801657 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:09.801620 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" Sep 09 02:20:09.802332 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:09.802314 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n2frh" Sep 09 02:20:27.004516 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:27.004482 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg"] Sep 09 02:20:27.009313 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:27.009296 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg" Sep 09 02:20:27.011737 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:27.011712 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 09 02:20:27.012476 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:27.012451 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-r6hw7\"" Sep 09 02:20:27.012476 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:27.012475 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 09 02:20:27.012621 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:27.012454 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\"" Sep 09 02:20:27.016406 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:27.016385 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg"] Sep 09 02:20:27.091811 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:27.091775 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9fcd042-80c6-41b0-9568-f328ae73be56-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg\" (UID: \"a9fcd042-80c6-41b0-9568-f328ae73be56\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg" Sep 09 02:20:27.091981 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:27.091819 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9fcd042-80c6-41b0-9568-f328ae73be56-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg\" (UID: \"a9fcd042-80c6-41b0-9568-f328ae73be56\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg" Sep 09 02:20:27.091981 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:27.091862 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knzcl\" (UniqueName: \"kubernetes.io/projected/a9fcd042-80c6-41b0-9568-f328ae73be56-kube-api-access-knzcl\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg\" (UID: \"a9fcd042-80c6-41b0-9568-f328ae73be56\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg" Sep 09 02:20:27.091981 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:27.091880 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9fcd042-80c6-41b0-9568-f328ae73be56-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg\" (UID: \"a9fcd042-80c6-41b0-9568-f328ae73be56\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg" Sep 09 02:20:27.091981 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:27.091951 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9fcd042-80c6-41b0-9568-f328ae73be56-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg\" (UID: \"a9fcd042-80c6-41b0-9568-f328ae73be56\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg" Sep 09 02:20:27.092161 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:27.091985 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9fcd042-80c6-41b0-9568-f328ae73be56-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg\" (UID: \"a9fcd042-80c6-41b0-9568-f328ae73be56\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg" Sep 09 02:20:27.192723 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:27.192684 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knzcl\" (UniqueName: \"kubernetes.io/projected/a9fcd042-80c6-41b0-9568-f328ae73be56-kube-api-access-knzcl\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg\" (UID: \"a9fcd042-80c6-41b0-9568-f328ae73be56\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg" Sep 09 02:20:27.192895 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:27.192728 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9fcd042-80c6-41b0-9568-f328ae73be56-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg\" (UID: \"a9fcd042-80c6-41b0-9568-f328ae73be56\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg" Sep 09 02:20:27.192895 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:27.192771 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9fcd042-80c6-41b0-9568-f328ae73be56-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg\" (UID: \"a9fcd042-80c6-41b0-9568-f328ae73be56\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg" Sep 09 02:20:27.192895 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:27.192804 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9fcd042-80c6-41b0-9568-f328ae73be56-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg\" (UID: \"a9fcd042-80c6-41b0-9568-f328ae73be56\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg" Sep 09 02:20:27.192895 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:27.192846 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9fcd042-80c6-41b0-9568-f328ae73be56-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg\" (UID: \"a9fcd042-80c6-41b0-9568-f328ae73be56\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg" Sep 09 02:20:27.192895 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:27.192888 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9fcd042-80c6-41b0-9568-f328ae73be56-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg\" (UID: \"a9fcd042-80c6-41b0-9568-f328ae73be56\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg" Sep 09 02:20:27.193219 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:27.193192 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9fcd042-80c6-41b0-9568-f328ae73be56-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg\" (UID: \"a9fcd042-80c6-41b0-9568-f328ae73be56\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg" Sep 09 02:20:27.193219 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:27.193211 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9fcd042-80c6-41b0-9568-f328ae73be56-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg\" (UID: \"a9fcd042-80c6-41b0-9568-f328ae73be56\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg" Sep 09 02:20:27.193360 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:27.193337 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9fcd042-80c6-41b0-9568-f328ae73be56-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg\" (UID: \"a9fcd042-80c6-41b0-9568-f328ae73be56\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg" Sep 09 02:20:27.195177 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:27.195153 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9fcd042-80c6-41b0-9568-f328ae73be56-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg\" (UID: \"a9fcd042-80c6-41b0-9568-f328ae73be56\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg" Sep 09 02:20:27.195367 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:27.195347 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9fcd042-80c6-41b0-9568-f328ae73be56-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg\" (UID: \"a9fcd042-80c6-41b0-9568-f328ae73be56\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg" Sep 09 02:20:27.206627 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:27.206604 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knzcl\" (UniqueName: \"kubernetes.io/projected/a9fcd042-80c6-41b0-9568-f328ae73be56-kube-api-access-knzcl\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg\" (UID: \"a9fcd042-80c6-41b0-9568-f328ae73be56\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg" Sep 09 02:20:27.320739 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:27.320654 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg" Sep 09 02:20:27.594951 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:27.594926 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg"] Sep 09 02:20:27.597160 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:20:27.597128 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9fcd042_80c6_41b0_9568_f328ae73be56.slice/crio-2b01e97b9025763919c685c3f5ac3181108ddb0a1a79136f326e967ba1bc5ba2 WatchSource:0}: Error finding container 2b01e97b9025763919c685c3f5ac3181108ddb0a1a79136f326e967ba1bc5ba2: Status 404 returned error can't find the container with id 2b01e97b9025763919c685c3f5ac3181108ddb0a1a79136f326e967ba1bc5ba2 Sep 09 02:20:27.726761 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:27.726726 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg" event={"ID":"a9fcd042-80c6-41b0-9568-f328ae73be56","Type":"ContainerStarted","Data":"2b01e97b9025763919c685c3f5ac3181108ddb0a1a79136f326e967ba1bc5ba2"} Sep 09 02:20:28.733025 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:28.732987 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg" event={"ID":"a9fcd042-80c6-41b0-9568-f328ae73be56","Type":"ContainerStarted","Data":"03fa3640c48f7d195a7af74f913820e7bc922f41bc91efe0694ba3730eb58623"} Sep 09 02:20:28.754007 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:28.753942 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg" podStartSLOduration=1.79079996 podStartE2EDuration="2.753917704s" totalImagesPullingTime="963.117744ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:20:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:20:28.752187683 +0000 UTC m=+588.127219093" watchObservedRunningTime="2026-09-09 02:20:28.753917704 +0000 UTC m=+588.128949114" Sep 09 02:20:37.321076 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:37.321042 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg" Sep 09 02:20:37.321554 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:37.321116 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg" Sep 09 02:20:37.333525 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:37.333499 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg" Sep 09 02:20:37.789219 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:37.789185 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg" Sep 09 02:20:49.608193 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:49.607809 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b"] Sep 09 02:20:49.611503 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:49.611480 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b" Sep 09 02:20:49.613852 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:49.613829 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Sep 09 02:20:49.621823 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:49.621799 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b"] Sep 09 02:20:49.684249 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:49.684217 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3297cdae-92d5-4d7e-b3d7-bc041c99845a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b\" (UID: \"3297cdae-92d5-4d7e-b3d7-bc041c99845a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b" Sep 09 02:20:49.684411 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:49.684265 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3297cdae-92d5-4d7e-b3d7-bc041c99845a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b\" (UID: \"3297cdae-92d5-4d7e-b3d7-bc041c99845a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b" Sep 09 02:20:49.684411 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:49.684362 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3297cdae-92d5-4d7e-b3d7-bc041c99845a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b\" (UID: \"3297cdae-92d5-4d7e-b3d7-bc041c99845a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b" Sep 09 02:20:49.684487 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:49.684416 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3297cdae-92d5-4d7e-b3d7-bc041c99845a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b\" (UID: \"3297cdae-92d5-4d7e-b3d7-bc041c99845a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b" Sep 09 02:20:49.684487 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:49.684438 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xctt\" (UniqueName: \"kubernetes.io/projected/3297cdae-92d5-4d7e-b3d7-bc041c99845a-kube-api-access-8xctt\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b\" (UID: \"3297cdae-92d5-4d7e-b3d7-bc041c99845a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b" Sep 09 02:20:49.684487 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:49.684476 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3297cdae-92d5-4d7e-b3d7-bc041c99845a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b\" (UID: \"3297cdae-92d5-4d7e-b3d7-bc041c99845a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b" Sep 09 02:20:49.785603 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:49.785557 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3297cdae-92d5-4d7e-b3d7-bc041c99845a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b\" (UID: \"3297cdae-92d5-4d7e-b3d7-bc041c99845a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b" Sep 09 02:20:49.785603 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:49.785609 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3297cdae-92d5-4d7e-b3d7-bc041c99845a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b\" (UID: \"3297cdae-92d5-4d7e-b3d7-bc041c99845a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b" Sep 09 02:20:49.785823 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:49.785640 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3297cdae-92d5-4d7e-b3d7-bc041c99845a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b\" (UID: \"3297cdae-92d5-4d7e-b3d7-bc041c99845a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b" Sep 09 02:20:49.785823 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:49.785662 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xctt\" (UniqueName: \"kubernetes.io/projected/3297cdae-92d5-4d7e-b3d7-bc041c99845a-kube-api-access-8xctt\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b\" (UID: \"3297cdae-92d5-4d7e-b3d7-bc041c99845a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b" Sep 09 02:20:49.785823 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:49.785702 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3297cdae-92d5-4d7e-b3d7-bc041c99845a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b\" (UID: \"3297cdae-92d5-4d7e-b3d7-bc041c99845a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b" Sep 09 02:20:49.785823 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:49.785758 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3297cdae-92d5-4d7e-b3d7-bc041c99845a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b\" (UID: \"3297cdae-92d5-4d7e-b3d7-bc041c99845a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b" Sep 09 02:20:49.786215 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:49.786183 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3297cdae-92d5-4d7e-b3d7-bc041c99845a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b\" (UID: \"3297cdae-92d5-4d7e-b3d7-bc041c99845a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b" Sep 09 02:20:49.786215 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:49.786207 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3297cdae-92d5-4d7e-b3d7-bc041c99845a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b\" (UID: \"3297cdae-92d5-4d7e-b3d7-bc041c99845a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b" Sep 09 02:20:49.786476 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:49.786240 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3297cdae-92d5-4d7e-b3d7-bc041c99845a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b\" (UID: \"3297cdae-92d5-4d7e-b3d7-bc041c99845a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b" Sep 09 02:20:49.788085 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:49.788064 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3297cdae-92d5-4d7e-b3d7-bc041c99845a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b\" (UID: \"3297cdae-92d5-4d7e-b3d7-bc041c99845a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b" Sep 09 02:20:49.788251 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:49.788233 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3297cdae-92d5-4d7e-b3d7-bc041c99845a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b\" (UID: \"3297cdae-92d5-4d7e-b3d7-bc041c99845a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b" Sep 09 02:20:49.798022 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:49.797988 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xctt\" (UniqueName: \"kubernetes.io/projected/3297cdae-92d5-4d7e-b3d7-bc041c99845a-kube-api-access-8xctt\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b\" (UID: \"3297cdae-92d5-4d7e-b3d7-bc041c99845a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b" Sep 09 02:20:49.922681 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:49.922645 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b" Sep 09 02:20:50.194548 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:50.194516 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b"] Sep 09 02:20:50.197350 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:20:50.197326 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3297cdae_92d5_4d7e_b3d7_bc041c99845a.slice/crio-e7582ee33f426ca7e4122c78f752eff37fecc593533b6677011c069def43abcd WatchSource:0}: Error finding container e7582ee33f426ca7e4122c78f752eff37fecc593533b6677011c069def43abcd: Status 404 returned error can't find the container with id e7582ee33f426ca7e4122c78f752eff37fecc593533b6677011c069def43abcd Sep 09 02:20:50.198661 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:50.198644 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 02:20:50.839587 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:50.839538 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b" event={"ID":"3297cdae-92d5-4d7e-b3d7-bc041c99845a","Type":"ContainerStarted","Data":"08fb8610e24bfb49d9aa71e8d05d6582cedeee5bd5341d9b3e97ad3d5c975086"} Sep 09 02:20:50.839587 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:50.839594 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b" event={"ID":"3297cdae-92d5-4d7e-b3d7-bc041c99845a","Type":"ContainerStarted","Data":"e7582ee33f426ca7e4122c78f752eff37fecc593533b6677011c069def43abcd"} Sep 09 02:20:50.861976 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:50.861925 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b" podStartSLOduration=1.681957844 podStartE2EDuration="1.861907362s" totalImagesPullingTime="179.949518ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:20:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:20:50.86079034 +0000 UTC m=+610.235821774" watchObservedRunningTime="2026-09-09 02:20:50.861907362 +0000 UTC m=+610.236938765" Sep 09 02:20:59.923416 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:59.923378 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b" Sep 09 02:20:59.923865 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:59.923444 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b" Sep 09 02:20:59.936158 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:20:59.936130 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b" Sep 09 02:21:00.900235 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:21:00.900201 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b" Sep 09 02:21:02.002774 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:21:02.002735 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ks54c"] Sep 09 02:21:02.010912 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:21:02.010882 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ks54c" Sep 09 02:21:02.014312 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:21:02.014270 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Sep 09 02:21:02.015798 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:21:02.015768 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ks54c"] Sep 09 02:21:02.097203 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:21:02.097166 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16d4b993-79a2-4f8b-85c5-35dc568e50af-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ks54c\" (UID: \"16d4b993-79a2-4f8b-85c5-35dc568e50af\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ks54c" Sep 09 02:21:02.097378 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:21:02.097267 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16d4b993-79a2-4f8b-85c5-35dc568e50af-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ks54c\" (UID: \"16d4b993-79a2-4f8b-85c5-35dc568e50af\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ks54c" Sep 09 02:21:02.097378 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:21:02.097317 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16d4b993-79a2-4f8b-85c5-35dc568e50af-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ks54c\" (UID: \"16d4b993-79a2-4f8b-85c5-35dc568e50af\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ks54c" Sep 09 02:21:02.097378 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:21:02.097348 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16d4b993-79a2-4f8b-85c5-35dc568e50af-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ks54c\" (UID: \"16d4b993-79a2-4f8b-85c5-35dc568e50af\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ks54c" Sep 09 02:21:02.097481 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:21:02.097382 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16d4b993-79a2-4f8b-85c5-35dc568e50af-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ks54c\" (UID: \"16d4b993-79a2-4f8b-85c5-35dc568e50af\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ks54c" Sep 09 02:21:02.097481 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:21:02.097425 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82rs5\" (UniqueName: \"kubernetes.io/projected/16d4b993-79a2-4f8b-85c5-35dc568e50af-kube-api-access-82rs5\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ks54c\" (UID: \"16d4b993-79a2-4f8b-85c5-35dc568e50af\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ks54c" Sep 09 02:21:02.198364 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:21:02.198322 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16d4b993-79a2-4f8b-85c5-35dc568e50af-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ks54c\" (UID: \"16d4b993-79a2-4f8b-85c5-35dc568e50af\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ks54c" Sep 09 02:21:02.198548 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:21:02.198375 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16d4b993-79a2-4f8b-85c5-35dc568e50af-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ks54c\" (UID: \"16d4b993-79a2-4f8b-85c5-35dc568e50af\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ks54c" Sep 09 02:21:02.198548 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:21:02.198409 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16d4b993-79a2-4f8b-85c5-35dc568e50af-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ks54c\" (UID: \"16d4b993-79a2-4f8b-85c5-35dc568e50af\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ks54c" Sep 09 02:21:02.198548 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:21:02.198430 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82rs5\" (UniqueName: \"kubernetes.io/projected/16d4b993-79a2-4f8b-85c5-35dc568e50af-kube-api-access-82rs5\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ks54c\" (UID: \"16d4b993-79a2-4f8b-85c5-35dc568e50af\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ks54c" Sep 09 02:21:02.198548 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:21:02.198449 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16d4b993-79a2-4f8b-85c5-35dc568e50af-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ks54c\" (UID: \"16d4b993-79a2-4f8b-85c5-35dc568e50af\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ks54c" Sep 09 02:21:02.198548 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:21:02.198528 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16d4b993-79a2-4f8b-85c5-35dc568e50af-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ks54c\" (UID: \"16d4b993-79a2-4f8b-85c5-35dc568e50af\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ks54c" Sep 09 02:21:02.198951 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:21:02.198924 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16d4b993-79a2-4f8b-85c5-35dc568e50af-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ks54c\" (UID: \"16d4b993-79a2-4f8b-85c5-35dc568e50af\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ks54c" Sep 09 02:21:02.198951 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:21:02.198941 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16d4b993-79a2-4f8b-85c5-35dc568e50af-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ks54c\" (UID: \"16d4b993-79a2-4f8b-85c5-35dc568e50af\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ks54c" Sep 09 02:21:02.199169 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:21:02.198966 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16d4b993-79a2-4f8b-85c5-35dc568e50af-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ks54c\" (UID: \"16d4b993-79a2-4f8b-85c5-35dc568e50af\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ks54c" Sep 09 02:21:02.200709 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:21:02.200689 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16d4b993-79a2-4f8b-85c5-35dc568e50af-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ks54c\" (UID: \"16d4b993-79a2-4f8b-85c5-35dc568e50af\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ks54c" Sep 09 02:21:02.200808 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:21:02.200789 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16d4b993-79a2-4f8b-85c5-35dc568e50af-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ks54c\" (UID: \"16d4b993-79a2-4f8b-85c5-35dc568e50af\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ks54c" Sep 09 02:21:02.206425 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:21:02.206406 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82rs5\" (UniqueName: \"kubernetes.io/projected/16d4b993-79a2-4f8b-85c5-35dc568e50af-kube-api-access-82rs5\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ks54c\" (UID: \"16d4b993-79a2-4f8b-85c5-35dc568e50af\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ks54c" Sep 09 02:21:02.324125 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:21:02.324002 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ks54c" Sep 09 02:21:02.601482 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:21:02.601424 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ks54c"] Sep 09 02:21:02.604244 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:21:02.604217 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16d4b993_79a2_4f8b_85c5_35dc568e50af.slice/crio-d4cbec1e851dcd9d83250b93c53ef5e480b20406adbe658ba0a86cb346254804 WatchSource:0}: Error finding container d4cbec1e851dcd9d83250b93c53ef5e480b20406adbe658ba0a86cb346254804: Status 404 returned error can't find the container with id d4cbec1e851dcd9d83250b93c53ef5e480b20406adbe658ba0a86cb346254804 Sep 09 02:21:02.906361 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:21:02.906263 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ks54c" event={"ID":"16d4b993-79a2-4f8b-85c5-35dc568e50af","Type":"ContainerStarted","Data":"c94180053a8e36a6ee0590ae2e77cd62327729fd080ea4602bf6fb9b8ecb04e0"} Sep 09 02:21:02.906524 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:21:02.906360 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ks54c" event={"ID":"16d4b993-79a2-4f8b-85c5-35dc568e50af","Type":"ContainerStarted","Data":"d4cbec1e851dcd9d83250b93c53ef5e480b20406adbe658ba0a86cb346254804"} Sep 09 02:21:02.923043 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:21:02.922976 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ks54c" podStartSLOduration=1.744992303 podStartE2EDuration="1.92295729s" totalImagesPullingTime="177.964987ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:21:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:21:02.922159807 +0000 UTC m=+622.297191216" watchObservedRunningTime="2026-09-09 02:21:02.92295729 +0000 UTC m=+622.297988701" Sep 09 02:21:12.325125 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:21:12.325084 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ks54c" Sep 09 02:21:12.325596 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:21:12.325155 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ks54c" Sep 09 02:21:12.337705 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:21:12.337672 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ks54c" Sep 09 02:21:12.965082 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:21:12.965052 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ks54c" Sep 09 02:23:15.688341 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:15.688311 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-kzz89_b8176964-35e0-457b-b4d0-2719b5ebda10/manager/0.log" Sep 09 02:23:15.803857 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:15.803826 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-t4cmm_e8205051-6be7-45cc-9e3c-0182117ead59/manager/0.log" Sep 09 02:23:15.929713 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:15.929686 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5fb57c948c-p5nmx_1129f7d2-8c55-464c-9f3f-c5ebca809be6/manager/0.log" Sep 09 02:23:16.305303 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:16.305270 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b45446f48-4mb2l_6695767e-4dfb-4603-8e8c-b410fe2c61c9/manager/0.log" Sep 09 02:23:16.311816 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:16.311793 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b45446f48-4mb2l_6695767e-4dfb-4603-8e8c-b410fe2c61c9/copy-manifests/0.log" Sep 09 02:23:16.896327 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:16.896297 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6964769c6-8j58j_2eb4ecf0-eeb0-4bd9-995a-252aead22faa/maas-api/0.log" Sep 09 02:23:17.013070 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:17.013031 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-ccbtf_28ec02cd-97b1-4007-a16c-41640fb8c7d5/postgres/0.log" Sep 09 02:23:17.908324 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:17.908293 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-plmhn_78224fa0-c4b4-4bc5-8208-359681e17c53/manager/0.log" Sep 09 02:23:18.037488 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:18.037453 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-5hm9l_e99b236b-e562-47ad-b8e9-892f95166806/manager/0.log" Sep 09 02:23:18.147775 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:18.147748 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-mfs22_8ea7a2f1-03b4-4665-8286-5ef11d4eb9df/kuadrant-console-plugin/0.log" Sep 09 02:23:18.274673 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:18.274643 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-npvzz_917231e6-d141-4de0-9f8c-f5c648ddaba1/manager/0.log" Sep 09 02:23:18.513275 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:18.513240 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-bhxf5_a7b1d8fd-ca1d-40f2-ab83-9a1621c3421c/manager/0.log" Sep 09 02:23:19.136791 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:19.136760 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-67698fd555-d842h_e9daf8a4-d9f9-4036-98eb-6d8d7feef5dc/kube-auth-proxy/0.log" Sep 09 02:23:19.368425 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:19.368389 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-n2frh_adc17dae-c69d-4f6a-a944-c7bda3c3f681/istio-proxy/1.log" Sep 09 02:23:19.491112 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:19.491079 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-xjqc8_4be4d3ed-a3d7-4841-8fc2-442ec813d2b2/payload-pre-processing/0.log" Sep 09 02:23:20.299932 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:20.299900 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-2rvsg_a9fcd042-80c6-41b0-9568-f328ae73be56/main/0.log" Sep 09 02:23:20.515701 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:20.515665 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fkrd5b_3297cdae-92d5-4d7e-b3d7-bc041c99845a/main/0.log" Sep 09 02:23:20.639829 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:20.639745 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-ks54c_16d4b993-79a2-4f8b-85c5-35dc568e50af/main/0.log" Sep 09 02:23:28.927623 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:28.927592 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-mzt99_0bf6e8eb-d36b-40d5-a3e3-9d9e148e0129/global-pull-secret-syncer/0.log" Sep 09 02:23:29.090316 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:29.090288 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-g8jhc_79f5d659-762b-4f17-be10-469a57ff032b/konnectivity-agent/0.log" Sep 09 02:23:29.141248 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:29.141219 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-218.ec2.internal_fe8949b5a0140471a9c7b59c781130ce/haproxy/0.log" Sep 09 02:23:33.505319 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:33.505238 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-plmhn_78224fa0-c4b4-4bc5-8208-359681e17c53/manager/0.log" Sep 09 02:23:33.554077 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:33.554045 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-5hm9l_e99b236b-e562-47ad-b8e9-892f95166806/manager/0.log" Sep 09 02:23:33.581543 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:33.581518 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-mfs22_8ea7a2f1-03b4-4665-8286-5ef11d4eb9df/kuadrant-console-plugin/0.log" Sep 09 02:23:33.619025 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:33.618984 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-npvzz_917231e6-d141-4de0-9f8c-f5c648ddaba1/manager/0.log" Sep 09 02:23:33.683346 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:33.683310 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-bhxf5_a7b1d8fd-ca1d-40f2-ab83-9a1621c3421c/manager/0.log" Sep 09 02:23:35.539029 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:35.538997 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-4fr2r_a66f722e-21be-4b13-9eb4-acef95b3b4ae/kube-state-metrics/0.log" Sep 09 02:23:35.569326 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:35.569296 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-4fr2r_a66f722e-21be-4b13-9eb4-acef95b3b4ae/kube-rbac-proxy-main/0.log" Sep 09 02:23:35.594769 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:35.594742 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-4fr2r_a66f722e-21be-4b13-9eb4-acef95b3b4ae/kube-rbac-proxy-self/0.log" Sep 09 02:23:35.891485 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:35.891463 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rfctf_235e6e09-3eb3-4733-896d-c5245a0415f2/node-exporter/0.log" Sep 09 02:23:35.920742 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:35.920707 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rfctf_235e6e09-3eb3-4733-896d-c5245a0415f2/kube-rbac-proxy/0.log" Sep 09 02:23:35.940265 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:35.940241 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rfctf_235e6e09-3eb3-4733-896d-c5245a0415f2/init-textfile/0.log" Sep 09 02:23:36.081647 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:36.081621 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8e358331-a769-4146-a574-f9642e8e9705/prometheus/0.log" Sep 09 02:23:36.121781 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:36.121757 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8e358331-a769-4146-a574-f9642e8e9705/config-reloader/0.log" Sep 09 02:23:36.160941 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:36.160863 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8e358331-a769-4146-a574-f9642e8e9705/thanos-sidecar/0.log" Sep 09 02:23:36.197410 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:36.197368 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8e358331-a769-4146-a574-f9642e8e9705/kube-rbac-proxy-web/0.log" Sep 09 02:23:36.235915 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:36.235886 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8e358331-a769-4146-a574-f9642e8e9705/kube-rbac-proxy/0.log" Sep 09 02:23:36.259562 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:36.259535 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8e358331-a769-4146-a574-f9642e8e9705/kube-rbac-proxy-thanos/0.log" Sep 09 02:23:36.287619 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:36.287589 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8e358331-a769-4146-a574-f9642e8e9705/init-config-reloader/0.log" Sep 09 02:23:38.830696 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:38.830659 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7qrgf/perf-node-gather-daemonset-q5nf9"] Sep 09 02:23:38.834162 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:38.834142 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7qrgf/perf-node-gather-daemonset-q5nf9" Sep 09 02:23:38.836595 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:38.836571 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-7qrgf\"/\"default-dockercfg-4fk4x\"" Sep 09 02:23:38.837410 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:38.837389 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7qrgf\"/\"kube-root-ca.crt\"" Sep 09 02:23:38.837506 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:38.837389 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7qrgf\"/\"openshift-service-ca.crt\"" Sep 09 02:23:38.843483 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:38.843458 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7qrgf/perf-node-gather-daemonset-q5nf9"] Sep 09 02:23:38.907649 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:38.907613 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d2fa7bd0-4755-4363-815c-151e17f6ac53-sys\") pod \"perf-node-gather-daemonset-q5nf9\" (UID: \"d2fa7bd0-4755-4363-815c-151e17f6ac53\") " pod="openshift-must-gather-7qrgf/perf-node-gather-daemonset-q5nf9" Sep 09 02:23:38.907649 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:38.907649 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnf26\" (UniqueName: \"kubernetes.io/projected/d2fa7bd0-4755-4363-815c-151e17f6ac53-kube-api-access-cnf26\") pod \"perf-node-gather-daemonset-q5nf9\" (UID: \"d2fa7bd0-4755-4363-815c-151e17f6ac53\") " pod="openshift-must-gather-7qrgf/perf-node-gather-daemonset-q5nf9" Sep 09 02:23:38.907857 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:38.907671 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d2fa7bd0-4755-4363-815c-151e17f6ac53-proc\") pod \"perf-node-gather-daemonset-q5nf9\" (UID: \"d2fa7bd0-4755-4363-815c-151e17f6ac53\") " pod="openshift-must-gather-7qrgf/perf-node-gather-daemonset-q5nf9" Sep 09 02:23:38.907857 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:38.907773 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d2fa7bd0-4755-4363-815c-151e17f6ac53-podres\") pod \"perf-node-gather-daemonset-q5nf9\" (UID: \"d2fa7bd0-4755-4363-815c-151e17f6ac53\") " pod="openshift-must-gather-7qrgf/perf-node-gather-daemonset-q5nf9" Sep 09 02:23:38.907923 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:38.907872 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d2fa7bd0-4755-4363-815c-151e17f6ac53-lib-modules\") pod \"perf-node-gather-daemonset-q5nf9\" (UID: \"d2fa7bd0-4755-4363-815c-151e17f6ac53\") " pod="openshift-must-gather-7qrgf/perf-node-gather-daemonset-q5nf9" Sep 09 02:23:39.008408 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:39.008371 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d2fa7bd0-4755-4363-815c-151e17f6ac53-sys\") pod \"perf-node-gather-daemonset-q5nf9\" (UID: \"d2fa7bd0-4755-4363-815c-151e17f6ac53\") " pod="openshift-must-gather-7qrgf/perf-node-gather-daemonset-q5nf9" Sep 09 02:23:39.008408 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:39.008421 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnf26\" (UniqueName: \"kubernetes.io/projected/d2fa7bd0-4755-4363-815c-151e17f6ac53-kube-api-access-cnf26\") pod \"perf-node-gather-daemonset-q5nf9\" (UID: \"d2fa7bd0-4755-4363-815c-151e17f6ac53\") " pod="openshift-must-gather-7qrgf/perf-node-gather-daemonset-q5nf9" Sep 09 02:23:39.008623 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:39.008446 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d2fa7bd0-4755-4363-815c-151e17f6ac53-proc\") pod \"perf-node-gather-daemonset-q5nf9\" (UID: \"d2fa7bd0-4755-4363-815c-151e17f6ac53\") " pod="openshift-must-gather-7qrgf/perf-node-gather-daemonset-q5nf9" Sep 09 02:23:39.008623 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:39.008489 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d2fa7bd0-4755-4363-815c-151e17f6ac53-podres\") pod \"perf-node-gather-daemonset-q5nf9\" (UID: \"d2fa7bd0-4755-4363-815c-151e17f6ac53\") " pod="openshift-must-gather-7qrgf/perf-node-gather-daemonset-q5nf9" Sep 09 02:23:39.008623 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:39.008520 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d2fa7bd0-4755-4363-815c-151e17f6ac53-sys\") pod \"perf-node-gather-daemonset-q5nf9\" (UID: \"d2fa7bd0-4755-4363-815c-151e17f6ac53\") " pod="openshift-must-gather-7qrgf/perf-node-gather-daemonset-q5nf9" Sep 09 02:23:39.008623 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:39.008587 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d2fa7bd0-4755-4363-815c-151e17f6ac53-proc\") pod \"perf-node-gather-daemonset-q5nf9\" (UID: \"d2fa7bd0-4755-4363-815c-151e17f6ac53\") " pod="openshift-must-gather-7qrgf/perf-node-gather-daemonset-q5nf9" Sep 09 02:23:39.008749 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:39.008625 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d2fa7bd0-4755-4363-815c-151e17f6ac53-lib-modules\") pod \"perf-node-gather-daemonset-q5nf9\" (UID: \"d2fa7bd0-4755-4363-815c-151e17f6ac53\") " pod="openshift-must-gather-7qrgf/perf-node-gather-daemonset-q5nf9" Sep 09 02:23:39.008749 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:39.008653 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d2fa7bd0-4755-4363-815c-151e17f6ac53-podres\") pod \"perf-node-gather-daemonset-q5nf9\" (UID: \"d2fa7bd0-4755-4363-815c-151e17f6ac53\") " pod="openshift-must-gather-7qrgf/perf-node-gather-daemonset-q5nf9" Sep 09 02:23:39.008813 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:39.008770 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d2fa7bd0-4755-4363-815c-151e17f6ac53-lib-modules\") pod \"perf-node-gather-daemonset-q5nf9\" (UID: \"d2fa7bd0-4755-4363-815c-151e17f6ac53\") " pod="openshift-must-gather-7qrgf/perf-node-gather-daemonset-q5nf9" Sep 09 02:23:39.017364 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:39.017338 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnf26\" (UniqueName: \"kubernetes.io/projected/d2fa7bd0-4755-4363-815c-151e17f6ac53-kube-api-access-cnf26\") pod \"perf-node-gather-daemonset-q5nf9\" (UID: \"d2fa7bd0-4755-4363-815c-151e17f6ac53\") " pod="openshift-must-gather-7qrgf/perf-node-gather-daemonset-q5nf9" Sep 09 02:23:39.145474 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:39.145382 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7qrgf/perf-node-gather-daemonset-q5nf9" Sep 09 02:23:39.416140 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:39.416064 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7qrgf/perf-node-gather-daemonset-q5nf9"] Sep 09 02:23:39.419984 ip-10-0-128-218 kubenswrapper[2561]: W0909 02:23:39.419956 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd2fa7bd0_4755_4363_815c_151e17f6ac53.slice/crio-2e9936bee37e11f3c131744f70e86ac9089b48f067d9c08658bed18c1e65ba2e WatchSource:0}: Error finding container 2e9936bee37e11f3c131744f70e86ac9089b48f067d9c08658bed18c1e65ba2e: Status 404 returned error can't find the container with id 2e9936bee37e11f3c131744f70e86ac9089b48f067d9c08658bed18c1e65ba2e Sep 09 02:23:39.648062 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:39.647992 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7qrgf/perf-node-gather-daemonset-q5nf9" event={"ID":"d2fa7bd0-4755-4363-815c-151e17f6ac53","Type":"ContainerStarted","Data":"8b53bc4fd3d06786a17896cc46a91d5c50652ca1d102edacf607ada75a907aef"} Sep 09 02:23:39.648269 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:39.648075 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7qrgf/perf-node-gather-daemonset-q5nf9" event={"ID":"d2fa7bd0-4755-4363-815c-151e17f6ac53","Type":"ContainerStarted","Data":"2e9936bee37e11f3c131744f70e86ac9089b48f067d9c08658bed18c1e65ba2e"} Sep 09 02:23:39.648269 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:39.648104 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-7qrgf/perf-node-gather-daemonset-q5nf9" Sep 09 02:23:39.665546 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:39.665475 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-7qrgf/perf-node-gather-daemonset-q5nf9" podStartSLOduration=1.6654544580000001 podStartE2EDuration="1.665454458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 02:23:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:23:39.665185487 +0000 UTC m=+779.040216930" watchObservedRunningTime="2026-09-09 02:23:39.665454458 +0000 UTC m=+779.040485868" Sep 09 02:23:41.433104 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:41.433071 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vpmqs_7ec1c2f7-97f8-463a-be11-27db70b7f648/dns/0.log" Sep 09 02:23:41.463803 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:41.463780 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vpmqs_7ec1c2f7-97f8-463a-be11-27db70b7f648/kube-rbac-proxy/0.log" Sep 09 02:23:41.545627 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:41.545600 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wvtrf_55dc8777-be96-4d4f-9f9f-8180a1e158fe/dns-node-resolver/0.log" Sep 09 02:23:42.234496 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:42.234459 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-z7f6v_84536d51-1f06-4aac-ba98-7773a0e01692/node-ca/0.log" Sep 09 02:23:43.305970 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:43.305943 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-67698fd555-d842h_e9daf8a4-d9f9-4036-98eb-6d8d7feef5dc/kube-auth-proxy/0.log" Sep 09 02:23:43.364545 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:43.364514 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-n2frh_adc17dae-c69d-4f6a-a944-c7bda3c3f681/istio-proxy/0.log" Sep 09 02:23:43.364696 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:43.364620 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-n2frh_adc17dae-c69d-4f6a-a944-c7bda3c3f681/istio-proxy/1.log" Sep 09 02:23:43.386343 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:43.386316 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-xjqc8_4be4d3ed-a3d7-4841-8fc2-442ec813d2b2/payload-pre-processing/0.log" Sep 09 02:23:44.135923 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:44.135892 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gbk4t_c3847832-aac1-4411-8629-57ac0ba9427b/serve-healthcheck-canary/0.log" Sep 09 02:23:44.684770 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:44.684743 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nk9tm_60c1d790-25c5-4fcc-8d2b-fe69e8f309ae/kube-rbac-proxy/0.log" Sep 09 02:23:44.709934 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:44.709907 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nk9tm_60c1d790-25c5-4fcc-8d2b-fe69e8f309ae/exporter/0.log" Sep 09 02:23:44.733259 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:44.733235 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nk9tm_60c1d790-25c5-4fcc-8d2b-fe69e8f309ae/extractor/0.log" Sep 09 02:23:45.663008 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:45.662982 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-7qrgf/perf-node-gather-daemonset-q5nf9" Sep 09 02:23:47.147151 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:47.147118 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-kzz89_b8176964-35e0-457b-b4d0-2719b5ebda10/manager/0.log" Sep 09 02:23:47.188935 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:47.188903 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-t4cmm_e8205051-6be7-45cc-9e3c-0182117ead59/manager/0.log" Sep 09 02:23:47.243943 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:47.243909 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5fb57c948c-p5nmx_1129f7d2-8c55-464c-9f3f-c5ebca809be6/manager/0.log" Sep 09 02:23:47.403286 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:47.403209 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b45446f48-4mb2l_6695767e-4dfb-4603-8e8c-b410fe2c61c9/manager/0.log" Sep 09 02:23:47.428878 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:47.428853 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b45446f48-4mb2l_6695767e-4dfb-4603-8e8c-b410fe2c61c9/copy-manifests/0.log" Sep 09 02:23:49.217354 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:49.217279 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-67698fd555-d842h_e9daf8a4-d9f9-4036-98eb-6d8d7feef5dc/kube-auth-proxy/0.log" Sep 09 02:23:49.259916 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:49.259891 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-n2frh_adc17dae-c69d-4f6a-a944-c7bda3c3f681/istio-proxy/0.log" Sep 09 02:23:49.259916 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:49.259927 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-n2frh_adc17dae-c69d-4f6a-a944-c7bda3c3f681/istio-proxy/1.log" Sep 09 02:23:49.273642 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:49.273615 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-xjqc8_4be4d3ed-a3d7-4841-8fc2-442ec813d2b2/payload-pre-processing/0.log" Sep 09 02:23:50.617969 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:50.617881 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-67698fd555-d842h_e9daf8a4-d9f9-4036-98eb-6d8d7feef5dc/kube-auth-proxy/0.log" Sep 09 02:23:50.655030 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:50.654986 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-n2frh_adc17dae-c69d-4f6a-a944-c7bda3c3f681/istio-proxy/1.log" Sep 09 02:23:50.655030 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:50.655003 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-n2frh_adc17dae-c69d-4f6a-a944-c7bda3c3f681/istio-proxy/0.log" Sep 09 02:23:50.667647 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:50.667624 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-xjqc8_4be4d3ed-a3d7-4841-8fc2-442ec813d2b2/payload-pre-processing/0.log" Sep 09 02:23:56.074177 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:56.074149 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5g4kk_85d4dd2e-a53d-455e-a542-9ce5ebd5b479/kube-multus-additional-cni-plugins/0.log" Sep 09 02:23:56.098463 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:56.098435 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5g4kk_85d4dd2e-a53d-455e-a542-9ce5ebd5b479/egress-router-binary-copy/0.log" Sep 09 02:23:56.121501 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:56.121477 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5g4kk_85d4dd2e-a53d-455e-a542-9ce5ebd5b479/cni-plugins/0.log" Sep 09 02:23:56.144042 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:56.143993 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5g4kk_85d4dd2e-a53d-455e-a542-9ce5ebd5b479/bond-cni-plugin/0.log" Sep 09 02:23:56.167414 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:56.167352 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5g4kk_85d4dd2e-a53d-455e-a542-9ce5ebd5b479/routeoverride-cni/0.log" Sep 09 02:23:56.236717 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:56.236684 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5g4kk_85d4dd2e-a53d-455e-a542-9ce5ebd5b479/whereabouts-cni-bincopy/0.log" Sep 09 02:23:56.262030 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:56.261995 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5g4kk_85d4dd2e-a53d-455e-a542-9ce5ebd5b479/whereabouts-cni/0.log" Sep 09 02:23:56.478289 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:56.478260 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-psv7l_7aaa0ef6-52d4-4878-9a87-8cc0da967919/kube-multus/0.log" Sep 09 02:23:56.534647 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:56.534617 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-8x9lh_a06a9649-a313-401a-80cb-c71fddc9d349/network-metrics-daemon/0.log" Sep 09 02:23:56.557697 ip-10-0-128-218 kubenswrapper[2561]: I0909 02:23:56.557660 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-8x9lh_a06a9649-a313-401a-80cb-c71fddc9d349/kube-rbac-proxy/0.log"